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Abstract 

There  are  currently  only  two  force  packages  under  consideration  for  Air 
Expeditionary  Force  (AEF)  deployment:  the  Eagle  package  and  the  Falcon  package. 
This  thesis  examines  these  and  three  other  force  packages  for  operational  feasibility. 

The  comparisons  made  between  the  five  packages  were  based  on  five  operational 
measures  and  two  which  describe  the  logistics  involved  with  deploying  each  package. 
The  operational  measures  and  one  logistics  measure  are  obtained  from  the  THUNDER 
simulation  software  package.  The  other  logistics  measure,  amoimt  of  short-tons  of  cargo 
required  to  support  each  package,  was  obtained  from  the  Logistics  Plans  office  at  HQ 
ACC. 

The  principal  contribution  is  a  methodology  for  modeling  and  analyzing  AEF 
packages  using  THUNDER  and  statistical  tools.  A  key  result  (based  on  two  notional 
scenarios)  was  that  adding  additional  F-16  aircraft  to  the  single  MDS  (all  F-16)  package 
had  a  negligible  impact  on  many  of  the  measures. 

Furthermore,  the  single  MDS  package  recorded  the  most  air-to-air  kills  and 
enemy  ground  targets  destroyed.  These  are  counterintuitive,  because  two  of  the  other 
packages  contained  equal  numbers  of  F-15Cs  and  F-15Es.  Note  that  these  results  should 
be  verified  by  using  an  actual  theater  scenario  for  THUNDER. 
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A  COMPARISON  OF  PROPOSED  AIR  EXPEDITIONARY  FORCE  PACKAGES 


USING  THE  THUNDER  CAMPAIGN  SIMULATION  PROGRAM 

I.  Introduction 

Background 

In  August  of  1997,  the  commander  of  the  Air  Combat  Command  (ACC),  General 
Richard  Hawley,  dedicated  the  Air  Expeditionary  Force  (AEF)  Battlelab  at  Mountain 
Home  Air  Force  Base,  Idaho.  General  Hawley  stated  that,  .  .the  mission  of  the  battlelab 
is  to  figure  out  ways  we  can  make  ourselves  lighter  and  more  agile.  This  will  help  get 
forces  deployed  to  where  they  are  needed  on  short  notice  with  less  airlift  than  we’ve 
needed  in  the  past  and  without  as  big  a  requirement  for  support  upon  arrival.”  (Mattson, 
1997:1). 

Brigadier  General  William  R.  Looney  III,  Commandant  of  the  Armed  Forces  Staff 
College,  wrote  an  article  entitled.  The  Air  Expeditionary  Force:  Taking  the  Air  Force 
into  the  Twenty-First  Century  which  appeared  in  the  Winter  1996  issue  of  the  Airpower 
Journal.  In  it,  he  describes  an  AEF  as  a  package  consisting  of  30  aircraft:  twelve  aircraft 
serving  in  an  air-to-air  role,  twelve  air-to-ground  (strike)  aircraft,  and  six  aircraft  to 
perform  a  suppression  of  enemy  air  defenses  (SEAD)  mission  (Looney,  1996:  6).  At  an 
AEF  conference  in  the  Spring  of  1998,  the  number  of  SEAD  aircraft  was  increased  to 
twelve,  bringing  the  size  of  the  entire  package  to  36  aircraft.  Also  at  the  conference,  the 
idea  of  including  heavy  bombers  fi-om  the  CONUS  in  the  package  was  proposed.  Taking 
off  from  their  home  stations,  B-ls  and/or  B-52s  could  rendezvous  with  the  strike  package 
and  increase  the  air-to-ground  capability  of  the  AEF  (Hoxie,  1998). 
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A  package  such  as  the  one  described  could  be  expected  to  generate  between  40 
and  60  sorties  per  day.  As  of  the  publication  date  of  General  Looney’s  article,  it  is 
estimated  that  approximately  1000  personnel  would  be  required  to  support  an  AEF.  This 
figure  would  increase  to  1,175  if  tanker  support  is  included.  Early  estimations  of  airlift 
required  placed  a  demand  of  fifty  to  sixty  C-141  equivalents  (Looney,  1996:  7-8). 

When  planning  for  an  AEF  deployment,  trade-offs  must  be  made  between 
operational  effectiveness  and  logistics  requirements.  The  more  diverse  and  capable  an  air 
package  is,  the  more  it  requires  in  terms  of  logistics  support.  Conversely,  a  force  mix 
which  does  not  take  advantage  of  its  full  range  of  combat  capability  is  sacrificing 
operational  output,  but  experiences  logistics  benefits. 

The  concept  of  an  AEF  is  different  from  the  common  deployment  of  a  squadron  in 
that  an  AEF  is  meant  to  be  short-term,  and  that  it  will  eliminate  the  current  90-  to  120- 
day  rotations  overseas.  By  the  year  2000,  it  is  hoped  that  the  idea  will  have  evolved  to 
the  point  where  two  of  the  projected  ten  AEFs  will  sit  on  what  is  similar  to  alert  status 
which  rotates  every  90  days.  This  cycle  will  repeat  approximately  every  15  months.  If  a 
tasking  comes  down  for  a  squadron  to  deploy,  members  plan  for  a  seven-day  operation 
(Katzaman,  98:1). 

During  this  time,  the  unit  provides  a  quick,  sustained,  initial  strike  capability, 
designed  to  halt  (or  at  least  delay)  the  advance  of  an  enemy.  While  the  AEF  is 
conducting  the  campaign,  other  units  back  in  the  states  are  mobilizing.  These  units  will 
bring  other  weapon  systems  to  the  theater  to  provide  a  diverse  package  of  airpower 
(Meserve,  1998). 
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It  is  expected  that  the  concept  of  an  AEF  will  take  advantage  of  the  ease  of 
mobility  associated  with  a  short-term  deployment,  and  be  able  to  conduct  effective 
operations.  Any  operational  short-comings  will  be  eliminated  when  the  main  force 
arrives  and  begins  to  conduct  operations  on  or  about  the  seventh  day.  At  that  time,  if 
deemed  necessary  by  warplanners,  force  packages  taking  full  advantage  of  the  flexibility 
of  current  air  power  will  conduct  operations.  In  this  manner,  the  AEF  will  experience  the 
logistics  benefits  of  a  “light”  deployment  and  conduct  operations  in  the  short-term,  and  a 
main  force  will  experience  the  operational  benefits  of  a  “heavy”  deployment. 

The  Intent  of  This  Research 

The  purpose  of  this  study  is  to  compare  the  operational  output  and  required 
logistics  support  for  the  following  four  air  packages  tmder  consideration  by  the  AEF 
Battlelab: 

Package  1 :  12  ea  F-15C  (Air-to-air) 

12  ea  F-15E  (Air-to-ground) 

12  ea  Block  50  F-16  (SEAD) 

Package  2:  1 2  ea  Block  30  F-1 6  (Air-to-air) 

12  ea  F-15E  (Air-to-ground) 

1 2  ea  Block  50  F- 1 6  (SEAD) 

Package  3:  12eaF-15C  (Air-to-air) 

1 2  ea  Block  40  F- 1 6  (Air-to-ground) 

12  ea  Block  50  F-16  (SEAD) 

Package  4;  12  ea  Block  30  F-16  (Air-to-air) 

12  ea  Block  40  F-16  (Air-to-ground) 

12  ea  Block  50  F-16  (SEAD) 

Package  four  is  of  particular  interest  because  it  is  the  only  force  mix  consisting  of  a  single 
mission  design  series  (MDS),  or  type  of  aircraft.  It  is  expected  that  this  package  will  be 


the  lightest  to  deploy  in  terms  of  short-tons  of  support  equipment  required.  This  is 
because  much  of  the  equipment  sent  to  support  a  given  squadron  in  the  package  could  be 
cross-utilized  by  other  squadrons  participating  in  the  AEF. 

The  all  F-16  (single  MDS)  package  is  included  in  the  force  mixes  under 
consideration  because  of  that  aircraft’s  ability  of  perform  all  missions  required  of  an 
AEF.  There  is  no  package  consisting  solely  of  F-1 5s  because  that  aircraft  does  not 
perform  the  SEAD  mission.  Although  it  can  carry  more  ordnance  than  an  F-16 
performing  the  same  mission  (therefore  requiring  fewer  sorties  to  attack  the  same  number 
of  targets),  this  one  shortcoming  eliminates  it  from  consideration  when  examining  the 
feasibility  of  a  single  MDS  AEF. 

If  it  turns  out  that  package  four  is  the  lightest  package  in  terms  of  support 
equipment  required,  the  next  step  would  be  to  determine  how  many  aircraft  (and  how 
many  short-tons  of  support  equipment)  would  be  required  to  provide  the  same  measures 
of  combat  output  as  the  heavier  deployments.  This  is  a  logical  step,  since  the  F-16 
carries  less  ordnance  than  an  F-1 5  performing  the  same  mission.  The  expected  benefit 
comes  from  the  lighter  deployment,  but  effectiveness  is  sacrificed.  The  true  measure  of 
the  benefit  (and  aim  of  this  thesis)  is  to  determine  if  a  larger  number  of  F-1 6s  can 
compare  with  the  other  force  packages  in  terms  of  operational  output.  It  is  for  this  reason 
that  a  fifth  package  will  be  included  in  the  analysis: 

Package  5:  18  ea  Block  30  F-16  (Air-to-air) 

18  ea  Block  40  F-16  (Air-to-groimd) 

18  ea  Block  50  F-16  (SEAD) 
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Package  five  consists  of  eighteen  aircraft  in  each  mission  (squadron)  because  that 
number  is  the  next  incremental  increase  for  force  packages,  as  described  by  current 
planning  documents.  This  package  serves  as  a  method  of  performing  a  sensitivity 
analysis  to  the  results  of  both  the  operational  output  and  support  equipment  requirements. 
If  package  four  provides  the  least  combat  output  of  the  four  packages  previously  defined 
(but  still  experiences  the  logistics  benefits  of  a  single  MDS  deployment),  package  five  is 
an  attempt  to  determine  if  more  aircraft  in  a  single  MDS  deployment  can  approach  the 
combat  output  of  a  dual  MDS  AEF  with  fewer  aircraft. 

This  is  an  interesting  prospect  in  light  of  the  intuitive  idea  that  this  package  would 
also  experience  the  logistics  benefits  (if  any)  of  a  single  MDS  deployment.  Although  this 
force  mix  will  most  likely  require  more  support  than  package  four,  it  deserves 
consideration  in  this  analysis  due  to  its  expected  increase  in  combat  output. 

THUNDER  Campaign  Analysis  Package 

This  thesis  uses  the  THUNDER  theater  simulation  software  package  as  a  means  to 
compare  the  combat  effectiveness  of  each  package.  The  first  version  of  THUNDER  used 
by  the  Air  Force  was  TAC  THUNDER  in  1986.  The  version  used  in  this  study,  6.4,  was 
introduced  in  1996.  The  current  version  is  the  result  of  more  than  a  dozen  upgrades  to  the 
original  and  considers  more  elements  of  a  campaign  to  make  the  program  more  versatile. 
Improvements  incorporated  into  the  prior  version  to  create  6.4  include  a  more  active 
flight  path  generation  algorithm  to  avoid  surface  to  air  (SAM)  sites;  improved 
identification,  surveillance,  and  reconnaissance  (ISR)  fimctions;  utilization  of  satellites  in 
the  performance  of  (ISR)  missions;  and  more  interaction  between  supporting  carrier  task 
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forces  and  other  units  in  the  battlespace  (THUNDER  Analyst  Manual  [TAM],  Version 
6.4,  Vol  1:  1996:21). 

THUNDER  is  in  wide  use  by  the  United  States  Air  Force  analysis  community  as 

well  as  the  following  military  and  civilian  agencies: 

National  Defense  War  Gaming  and  Simulation  Center, 

United  Kingdom  Ministry  of  Defence  Directorate  of  Science  (Air), 

Royal  Air  Force  -  Cranwell  Department  of  Air  Warfare, 

Boeing  Military  Airplane  Division, 

Rockwell  International, 

French  Ministry  of  Defence  Bureau  de  Prospective  et  de  Recherche  Operationelle, 
Northrop-Grumman  Advanced  Technology  and  Development  Center,  and 
Institute  for  Defense  Analysis  (TAM,  Version  6.4,Vol  1;  25:  1996). 


Input  for  THUNDER  comes  from  over  80  different  input  data  files,  describing 
(among  other  things)  type  and  number  of  aircraft  in  the  theater,  weather  bands,  command 
and  control  measures,  air  defenses,  ground  targets,  and  probability  of  kill  (PK)  data  of 
munitions  preprogrammed  into  the  software.  The  THUNDER  package  includes  three 
preprogrammed  scenarios  entitled  Datasmall,  MacAir,  and  ME.  These  allow  the  novice 
user  to  become  familiar  with  the  simulation  package  (TAM,  Version  6.5,  Vol  3;  1996:  2). 

Input  data  for  the  preprogrammed  scenarios  consist  of  notional,  unclassified 
information  describing  the  probable  enemy  threat,  expected  allied  presence,  and  order  of 
battle  in  an  area.  Datasmall  and  MacAir  describe  a  European  theater,  whereas  ME 
describes  a  Middle  East  scenario.  The  information  preprogrammed  into  these  data  files 
are  best  estimates  for  each  parameter.  They  are  the  most  reliable  unclassified  data 
available  (TAM,  Version  6.5,  Vol  3;  1996:  2). 
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An  additional  database,  Storm,  is  included  in  the  version  maintained  by  AFIT  and 
is  based  on  a  Middle  East  scenario  with  data  from  Operation  DESERT  STORM.  Data  for 
past  AFIT  theses  have  been  based  on  either  the  ME  or  Storm  scenarios.  Upon  becoming 
familiar  with  THUNDER  using  the  preprogrammed  scenarios,  the  user  can  modify 
appropriate  input  data  files  to  reflect  the  situation  under  study.  In  this  study,  data 
programmed  into  THUNDER  were  the  proposed  AEF  packages  in  the  Storm  scenario. 
Output  from  the  five  air  packages  were  compared  to  get  an  idea  of  how  each  performed  in 
relation  to  the  other  four  in  terms  of  combat  effectiveness. 

Research  Questions 

1 .  What  is  the  ranking  of  the  five  AEF  packages  in  terms  of  combat  effectiveness 
and  support  required? 

It  is  expected  that  package  one  consisting  of  F-15Cs  in  the  air-to-air  role,  F-15Es 
serving  in  air-to-ground,  and  Block  50  F- 16s  performing  the  SEAD  function  will  have  the 
highest  combat  output  in  terms  of  enemy  ground  targets  destroyed,  enemy  aircraft 
destroyed,  etc.  The  second  and  third  packages  are  expected  to  be  similar  to  each  other  in 
terms  of  effectiveness.  Finally,  the  forth  package,  consisting  solely  of  F-16s,  would 
likely  be  the  least  combat  effective. 

Constraints  on  airlift  make  the  logistics  involved  with  deploying  an  AEF  a 
primary  issue.  Not  only  is  volume  of  equipment  a  concern,  but  also  the  timeline  required 
to  meet  the  "lean,  light,  and  lethal"  goals  of  an  AEF  (Parsons,  1998).  In  terms  of  support 
equipment  required,  the  package  consisting  of  two  squadrons  of  the  larger,  twin-engine 
F-15  (package  one)  is  expected  to  require  more  support  equipment  and  spare  parts  than 
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any  other  package.  Packages  two  and  three  are  expected  to  be  somewhat  lighter  and 
similar  to  each  other  due  to  the  fact  that  both  packages  consist  of  two  squadrons  of  F- 16s 
and  one  squadron  of  F- 15s.  Finally,  the  lightest  package  is  expected  to  be  package  four, 
the  single  MDS  package  due  to  cross-utilization  possibilities  of  spare  parts  and  support 
equipment. 

2.  Would  increasing  the  number  of  aircraft  in  the  lightest  package  increase  that 
package ‘s  combat  output? 

Increasing  the  number  of  aircraft  in  any  package  should  increase  that  package’s 
output.  If  the  original  package  is  light  to  begin  with,  the  increase  in  its  support 
requirement  for  additional  aircraft  may  be  acceptable.  Package  five  attempts  to  answer 
this  question.  If  the  support  requirements  of  these  additional  aircraft  is  acceptable,  it 
may  be  beneficial  to  accept  the  increased  support  to  achieve  a  greater  combat  output. 

3.  Is  the  relative  ranking  of  combat  effectiveness  of  each  air  package  constant 
throughout  different  threat  scenarios? 

Although  a  given  package  my  be  less  effective  than  the  others  in  the 
preprogrammed  Storm  scenario,  an  attempt  will  be  made  to  find  a  scenario  which 
reorders  the  ranking  of  the  packages.  An  alternate  scenario  will  be  programmed  in  which 
different  mixes  of  enemy  aircraft  and  surface  threats  exist  in  an  attempt  to  reorder  the 
effectiveness  rating  of  the  packages  under  study. 

Organization 

The  Literature  Review  reveals  past  research  into  areas  similar  to  these  research 
questions.  As  mentioned  before,  several  theses  have  been  written  based  on  data  obtained 


8 


from  THUNDER.  These  are  discussed  in  the  review,  as  well  as  other  works  addressing 
support  requirements  from  different  angles.  General  Looney’s  article  explaining  the  AEF 
concept  has  already  been  mentioned.  The  literature  review  discusses  other  articles 
relevant  to  logistics  support  and  combat  effectiveness  of  AEFs. 

Chapter  III  explains  the  methodology  employed  to  answer  the  research  questions. 
The  manipulation  of  appropriate  THUNDER  input  data  files  is  discussed  and  justification 
for  each  modification  is  provided.  Default  values  and  the  baseline  scenario  in 
THUNDER  are  explained  in  the  chapter,  as  well  as  measures  of  effectiveness  (MOEs) 
used  as  bases  of  comparison  between  the  five  air  packages. 

Analysis  of  the  MOEs  from  THUNDER  is  provided  in  Chapter  IV.  Statistical 
tools  are  used  to  rank  the  air  packages  in  terms  of  combat  output  and  support 
requirements. 

Chapter  V  consists  of  conclusions  and  recommendations,  and  contains  a  summary 
of  the  findings  and  answers  the  research  questions  provided  earlier. 

Limitations 

It  was  mentioned  earlier  that  the  Storm  scenario  is  a  notional  database.  This 
characteristic  will  affect  the  output.  However,  the  methodology  and  analysis  are 
appropriate  regardless  of  classification.  This  limitation  will  be  addressed  as  an  avenue 
for  further  research. 

Another  limitation  of  this  research  is  that  although  data  describing  short-ton 
requirements  are  given,  the  amount  of  support  persoimel  required  for  each  package  is  not 
included  as  a  measure  for  comparison.  Persormel  requirements  are  not  examined  because 
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of  the  time  needed  to  perform  a  proper  analysis  of  available  logistics  data.  Therefore, 
logistics  requirements  are  only  based  on  estimates  for  the  amount  of  short-tons  required 
for  each  package. 

Finally,  only  those  aircraft  weapons  configurations  included  in  the  original  Storm 
database  (as  they  appear  in  the  typeac.dat  data  file)  were  used,  (typeac.dat  and  other  files 
are  discussed  in  the  Methodology  section.)  This  was  due  to  the  classification  constraint. 
No  classified  weapons  configurations  are  included. 
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11.  Literature  Review 


This  chapter  reviews  the  literature  available  on  the  topic  the  AEF  and  the 
THUNDER  simulation  package.  Previously  accomplished  theses  employing  THUNDER 
are  of  particular  interest  due  to  the  fact  that  they  provide  insight  of  the  utility  of 
THUNDER  beyond  its  obvious  use  as  a  war  planning  tool.  These  studies  also  assist  the 
novice  TETUNDER  user  in  how  to  use  the  software  to  get  the  desired  information. 
Appropriate  input  data  file  manipulation  is  essential  to  effective  use  of  the  package.  This 
comes  in  large  part  from  knowing  the  information  that  each  data  file  contains,  as  well  as 
how  to  interpret  the  data  contained  in  the  files. 

Once  the  analyst  becomes  familiar  with  data  in  the  files,  they  can  then  begin  to 
make  minor  manipulations  to  the  files  with  the  aim  of  discovering  how  each  seemingly 
insignificant  piece  of  data  impacts  the  entire  set  of  output  metrics.  From  this  type  of 
experimentation,  the  analyst  will  begin  to  understand  how  THUNDER  reads  the  input 
data  files  and  interprets  that  data  to  create  the  scenario  and  conduct  the  war. 
Understanding  of  THUNDER’S  operation  is  a  great  facilitator  in  effectively  using  the 
software. 

Prior  AEF  Research 

It  is  expected  that  the  most  combat-capable  package  would  be  package  one  as 
described  earlier.  But  the  effectiveness  of  an  AEF  consisting  solely  of  F- 16s  is  also  of 
interest  in  this  study.  The  idea  of  a  single  MDS  AEF  is  fairly  new  to  offensive  strategy. 
A  review  of  the  DTIC  database,  the  EBSCO  search  software,  and  previous  theses 
revealed  no  prior  research  in  this  area,  although  there  are  several  journal  articles  and 
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research  papers  on  the  concept  of  an  AEF.  These  provided  a  basis  for  background 
research  and  introduced  the  idea  of  an  AEF. 

The  previously  mentioned  article  by  Brigadier  General  Looney  provides  excellent 
insight  into  the  notion  of  an  AEF.  Although  the  concept  at  the  time  of  publication  was 
still  in  its  infancy  (its  appearance  in  the  Journal  preceded  the  AEF  Battlelab  dedication 
by  almost  one  year),  its  principles  are  in  large  part  still  true.  The  only  significant 
difference  between  the  scenario  General  Looney  described  and  the  current  design  is  the 
addition  of  six  SEAD  (Block  50  F-16)  aircraft  (Hoxie:  1998). 

John  A.  Tirpac,  Senior  Editor  of  Air  Force  Magazine,  contributed  an  zirticle  to  the 
June  1998  issue  in  which  he  explains  future  roles  for  the  B-1 .  In  the  article,  Colonel 
Glenn  Spears,  commander  of  the  28th  Operations  Group  at  Ellsworth  AFB,  explains  the 
preferred  role  of  the  B-1  as  part  of  a  package.  The  package  is  described  as  consisting  of 
“F-15s  for  fighter  cap  (combat  air  patrol),  F-16CJs  (Block  50  F-16s)  with  HARM 
missiles  for  Suppression  of  Enemy  Air  Defenses,  some  F-16s  as  bomb  droppers,  and 
some  F-15Es  for  precision  weapons  drop... ’’(Tirpac,  1998:  30).  This  package  described 
by  Colonel  Spears  resembles  the  packages  under  investigation  in  the  study. 

First  Lieutenant  Tam  Vo’s  Exploratory  Analysis  of  the  Deployment  Feasibility  of 
United  States  Air  Force  Air  Expeditionary  Forces  thesis  describes  the  Navy’s  carrier 
battle  group  and  the  Marine  Expeditionary  Force,  concepts  on  which  the  AEF  is  based. 
She  also  illustrates  the  sequence  of  events  on  the  48-hour  timeline  which  describes  the 
deployment  schedule  of  an  AEF.  The  timeline  begins  with  the  warning  order  and  ends 
with  bombs  on  target  (Vo,  1997:  43).  Lt  Vo  also  states  the  difficulty  past  AEFs  had  in 
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meeting  this  goal.  The  idea  of  a  single  MDS  AEF  is  bom  from  the  challenges  associated 
with  deploying  such  a  diverse  package  as  an  AEF  would  require.  Lessons  learned  from 
the  first  four  AEF  deployments  are  also  given  by  Lt  Vo  as  a  guide  to  future  planning. 

In  addition  to  articles  and  past  theses,  there  have  been  several  briefings  presented 
by  personnel  from  the  AEF  Battlelab,  the  Air  Force  Logistics  Management  Agency 
(AFLMA),  and  the  RAND  Corporation  concerning  AEFs.  Captain  Robert  Vaughn  of 
AFLMA  presented  a  briefing  to  Lieutenant  General  Jumper,  Air  Force  Director  of  Air 
Operations  (AF/XO),  and  Lieutenant  General  Halin,  Air  Force  Director  for  Installations 
and  Logistics  (AF/IL),  in  July  of  1997  describing  the  lessons  learned  from  the  first  three 
AEFs.  Two  major  areas  of  concern  he  addresses  are  predeployment  planning  and 
political-military  relations.  The  difficulties  associated  with  predeployment  planning  were 
of  particular  concern  due  to  the  fact  that,  according  to  Captain  Vaughn,  lessons  learned 
from  AEFs  I  and  II  were  not  used  in  the  preparations  for  AEF  III  (Vaughn,  1997). 

Political -military  relations  come  into  play  when  considering  the  locations  of  the 
first  three  AEFs.  AEF  I  operated  out  of  Shaikh  Isa,  Bahrain;  AEF  II  from  Azraq,  Jordan; 
and  AEF  III  from  Doha,  Qatar.  Relations  between  the  host  country  and  visiting  military 
will  always  be  a  concern  when  operating  from  locations  such  as  these.  Someone 
unfamiliar  with  customs  of  the  host  coimtry  can  unknowingly  jeopardize  the  success  of 
not  only  the  current  mission  but  future  operations  as  well  (Vaughn,  1997). 

Captain  Vaughn  concludes  by  reiterating  the  fact  that  the  AEF  concept  as 
practiced  in  AEFs  I-III  is  sound.  He  attributes  most  of  the  problems  encountered  during 
AEF  III  to  poor  planning  and  not  using  lessons  learned  from  AEFs  I  and  II.  He  also 
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voices  a  concern  that  there  is  no  standardized  metric  to  measure  AEF  deployment 
efficiency  (Vaughn,  1997). 

Lieutenant  Colonel  Tony  Dronkers  from  the  Air  Staff  proposes  metrics  to 
measure  various  characteristics  of  interest  concerning  AEFs.  Of  primary  concern  is 
response  time.  This  is  a  measure  of  how  quickly  air  assets  can  arrive  in  theater. 

Response  time  differs  from  time  to  effectiveness  in  that  the  latter  describes  the  time  it 
takes  for  operations  to  begin  once  in  theater.  The  number  of  pallet  positions  is  widely 
accepted  now  as  the  standard  used  to  describe  how  much  equipment  is  used  to  support  the 
initial  arrival  of  a  unit  at  the  beginning  of  particular  deployment  (footprint).  Demand  pull 
is  used  to  describe  the  amount  of  equipment  (expressed  in  terms  of  pallets  per  day) 
needed  to  sustain  a  deployment  (logistics  pipeline)  (Dronkers,  1997). 

Lt  Col  Dronkers  also  explains  the  concept  of  regional  contingency  centers 
(RCCs).  These  are  staging  areas  from  which  AEFs  could  be  conducted  in  the  future. 
There  are  currently  four  centers  selected,  each  operating  on  a  1500-2000  nautical  mile 
radius.  The  one  in  the  Western  Hemisphere  would  operate  out  of  Roosevelt  Roads  and 
serve  the  southeastern  portion  of  North  America  and  northwestern  quarter  of  South 
America.  There  is  one  centered  in  Western  Europe  at  Moron,  Spain  and  could  cover  all 
of  Europe  and  the  northwestern  third  of  the  African  Continent.  Southwestern  and  South- 
central  Asia  would  be  covered  by  an  AEF  staging  from  Diego  Garcia.  AEFs  operating  in 
Eastern  and  Southeast  Asia  would  deploy  to  Guam  (Dronkers,  1997). 

The  geographic  location  of  each  of  these  areas  is  such  that  they  would  put  an  AEF 
within  striking  distance  of  areas  of  potential  hostilities.  Further,  Moron,  Diego  Garcia, 
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and  Guam  are  all  areas  from  which  heavy  bombers  have  operated  in  the  past  eind  could 
easily  serve  as  staging  bases  for  B-1  s  and  B-52s  serving  in  an  AEF.  An  AEF  operating 
from  Roosevelt  Roads  would  be  augmented  with  B-52s  from  Barksdale  AFB.  There  are 
also  four  more  proposed  RCCs  which  would  “plug  some  of  the  holes”  resulting  from  the 
network  of  current  RCCs.  The  result  of  adding  these  proposed  centers  is  a  continuous 
umbrella  of  AEF  coverage  beginning  in  North  America  and  continuing  unbroken  east  to 
Midway  Island  in  the  Pacific  Ocean  (Dronkers,  1997). 

In  a  briefing  by  Maj  Ernie  Eannarino  from  AF/XOCD  (the  office  responsible  for 
drafting  the  AEF  Air  Force  Instruction),  the  AEF  concept  was  stated  as  being  a  means  to 
“provide  regional  CINCs  with  rapid  and  responsive  air  and  space  power,  tailored  to  meet 
theater  specific  needs  across  the  spectrum  of  response  options  from  humanitarian  relief  to 
combat  operations. . .”  (Eannarino,  1997).  This  mention  of  humanitarian  relief  implies  the 
flexibility  of  the  AEF.  Whereas  combat  AEFs  would  be  conducted  in  large  measure  by 
ACC,  USAFE,  and  PACAF  (depending  on  location),  AMC  would  be  expected  to  play  a 
the  lead  role  in  a  humanitarian  AEF  (HAEF)  (Eannarino,  1997). 

Specific  strengths  of  AEFs  are  defined  to  be  speed,  strength,  and  flexibility. 

AEFs  are  designed  to  be  a  rapid  and  decisive  response  to  hostilities  anywhere  in  the 
world.  To  allies  of  the  United  States,  AEFs  represent  a  commitment  on  behalf  of  the  US 
to  share  the  risks  and  burdens  of  conflict.  To  our  adversaries,  they  represent  “a  credible 
and  flexible  deterrent”  (Eannarino,  1997).  In  this  manner,  AEFs  can  be  expected  to  play 
a  major  role  in  future  conflicts,  as  well  as  in  military  operations  other  than  war 
(MOOTW). 
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At  the  time  of  presentation  by  Major  Eannarino,  there  had  been  four  AEFs: 

AEF  I:  Oct-Dec  1995  (18  aircraft);  Shaikh  Isa,  Bahrain 

AEF II:  Apr-Jun  1996  (30  aircraft);  Azraq/Hasan,  Jordan 

AEF  III:  Jun-Aug  1996  (34  aircraft);  Doha  Int’l,  Qatar 

AEF  IV :  Feb-Jun  1997  (30  aircraft);  Doha  Int’l,  Qatar 

Note  that  dates  given  for  these  prior  AEFs  imply  durations  longer  than  the  time 
AEFs  are  designed  to  be  active.  The  AEF  phase  of  a  deployment  is  intended  to  last  seven 
days.  During  this  time,  the  build  up  for  the  main  force  is  under  way.  If  the  conflict 
continues,  the  main  force  will  augment  the  AEF.  Equipment  needed  for  the  full  three-  or 
four-month  deployment  of  the  first  four  AEFs  may  not  be  needed  for  a  seven-day  AEF. 

Challenges  associated  with  host  nation  agreements  are  again  a  concern  for  the  Air 
Operations  Office.  This  was  also  a  concern  mentioned  by  Captain  Vaughn  of  AFLMA. 
Benefits  of  an  AEF  over  traditional  deployments  include  a  reduction  in  “permanent  and 
obtrusive  presence”  of  United  States  forces  in  other  countries.  When  the  Air  Force 
becomes  capable  of  deploying  light,  the  need  for  standing  troops  overseas  will  decrease. 
This  would  help  reduce  the  tensions  between  the  United  States  and  host  nations 
(Eannarino,  1997). 

The  Air  Operations  Office  also  saw  the  need  to  create  an  “AEF  mindset”  in  the 
force.  This  includes  changing  the  paradigms  in  training  and  doctrine.  For  AEFs  to  be 
successful  and  become  a  viable  threat  to  potential  enemies,  more  emphasis  needs  to  be 
placed  on  the  logistics  involved  with  sending  1000  personnel  and  36  aircraft  anywhere  in 
the  world  and  being  operational  within  48  hours  of  the  warning  order.  This  requires  a 
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change  in  the  current  mindset  of  sending  equipment  unless  it  is  not  needed  to  one  of  not 
sending  equipment  unless  it  is  needed  (Eannarino,  1997). 

Robert  S.  Tripp  of  the  RAND  Corporation  presented  a  briefing  at  the  December 
1997  Implementation  Conference  entitled,  Leveraging  Logistics  to  Enhance  the 
Effectiveness  of  Strike  Air  Expeditionary  Forces.  The  briefing  addressed  innovative 
methods  of  supporting  AEFs.  Under  the  umbrella  of  supportability,  the  issue  of  fuel 
distribution  was  brought  to  light.  Currently,  POL  (petroleum,  oil,  and  lubricants;  here  the 
term  applies  to  fuel)  is  supplied  by  transporting  large  fuel  trucks  via  ten  C-141s.  An 
option  under  consideration  is  to  use  what  are  termed  “austere  hot  pits”.  These  are  simply 
large  underground  fuel  pits  at  the  flightline  of  the  deployed  location.  If  these  are  present 
at  the  deployed  location,  this  would  be  the  preferred  method  of  POL  transport  (Tripp, 
1997). 

Leasing  POL  trucks  from  host  nation  military  or  civilian  contractor  is  another 
option.  This  requires  a  good  relationship  with  the  host  nation,  a  point  that  was  already 
mentioned  as  being  essential  to  the  success  of  an  AEF.  Finally,  limiting  deployments  to 
locations  with  adequate  POL  distribution  systems  is  the  best  way  to  ensure  an 
uninterrupted  supply  of  fuel,  but  this  may  run  contrary  to  the  goal  of  flexibility  in  an 
AEF.  Securing  an  adequate  POL  distribution  system  in  the  four  current  RCCs  already 
discussed  goes  a  long  way  in  reducing  the  amoimt  of  supportability  needed  for  an  AEF 
(Tripp,  1997). 

Billeting  of  personnel  was  discussed  as  being  essential  to  mission  success.  One 
current  method  of  housing  troops  is  called  “harvest  falcon  housekeeping  sets”.  These  are 
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large  tent-like  structures,  but  with  a  more  rigid  frame.  Future  options  include  large, 
prepositioned  structures  at  the  likely  deployment  centers.  When  deploying  to  extremely 
austere  locations,  the  most  likely  avenue  of  billeting  and  messing  would  be  to  issue  tents 
and  meals,  ready  to  eat  (MREs)  to  the  personnel  participating  in  the  AEF.  Finally,  the 
least  flexible  option  is  to  limit  deployments  to  areas  with  adequate  quarters  (hotels, 
barracks,  warehouses,  etc.)  already  in  place  (Tripp,  1997). 

The  emphasis  on  good  military-host  nation  relations  was  implied  in  this  briefing 
also.  If  the  hosts  at  the  proposed  RCC  agree  to  allow  the  United  States  to  preposition 
fuel,  billeting,  and  munitions,  the  deployment  footprint  to  these  locations  would  be 
drastically  reduced. 

Prior  THUNDER  Research 

Another  topic  researched  was  the  THUNDER  simulation  software  package.  This 
was  of  particular  interest  in  this  effort  since  THUNDER  output  would  serve  as  the  data 
for  analysis.  Captain  Timothy  Webb  completed  a  study  entitled.  Analysis  of  THUNDER 
Combat  Simulation  Model.  Sponsored  by  the  Campaign  Analysis  Branch  of  the 
Aeronautical  Systems  Center,  Captain  Webb  attempted  to  determine  which  input 
parameters  affected  the  MOEs  he  chose  to  investigate  and  to  what  degree. 

Captain  Webb  explains  the  two  methods  of  determining  the  number  of 
replications  in  an  experiment.  The  first  method  is  called  fixed  sample  size.  Using  this 
method,  the  researcher  simply  determines  the  number  of  runs  necessary  to  achieve  the 
desired  significance  level  based  on  statistical  calculations.  This  method  is  acceptable  if 
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the  mean  and  varianee  of  the  parameters  under  study  experience  little  change  through  the 
replications  (Law  and  Kelton;  1991:  532). 

The  other  method  is  the  sequential  approach.  The  sequential  approach  adds  one 
replication  at  a  time  until  the  desired  confidence  interval  of  the  MOE  is  obtained.  The 
disadvantage  of  the  fixed  sample  approach,  Webb  explains,  is  that  the  operator  has  no 
control  over  the  width  of  the  resulting  confidence  interval.  A  disadvantage  associated 
with  the  sequential  method  is  that  the  variance  of  the  parameter  under  study  may  be  so 
large  that  the  number  of  runs  needed  to  obtain  the  desired  confidence  could  be 
prohibitively  large.  The  analyst  would  not  be  aware  of  the  large  variance  unless  periodic 
calculations  are  performed  (Webb,  1994:  3-1). 

Captain  Webb  ran  the  predetermined  number  of  replications  in  THUNDER  for 
each  scenario  and  used  three  different  methods  (the  Bayesian  approach,  the  Iterative 
approach,  and  Response  Surface  Methodology  [RMS])  for  determining  significance  of 
the  factors  imder  study.  Each  of  the  three  methods  resulted  in  a  second  order  regression 
equation  which  represented  the  MOE  under  study  mathematically.  The  regression 
equations  were  used  to  predict  the  given  MOE  to  within  90  percent  certainty.  Confidence 
intervals  for  each  parameter  were  established,  a  procedure  which  will  be  undertaken  in 
this  study  also. 

Captain  Webb  offers  an  equation  from  Law  and  Kelton  which  can  be  used  to 
determine  the  number  of  repetitions  needed  to  obtain  a  given  level  of  confidence  for  the 
MOE  imder  study.  He  also  uses  an  equation  to  determine  the  accuracy  of  the  models  he 
created  via  stepwise  regression.  The  term  used  to  describe  error  in  the  models  is  mean 
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absolute  percent  error  (MAPE)  and  is  calculated  for  each  of  the  three  methods  used  to 
determine  significance  of  factors  in  the  models.  Many  of  Captain  Webb’s  conclusions 
consist  of  comparisons  between  the  three  methods  of  factor  significance  determination 
(Webb,  1994:  5-3). 

Major  David  Davies  completed  a  thesis  entitled  Sensitivity  Analysis  of  the 
THUNDER  Combat  Simulation  Model  to  Command  and  Control  Inputs  Accomplished  in 
a  Parallel  Environment.  Major  Davies  modified  those  input  files  which  included 
command  and  control  (C2)  parameters,  and  compared  how  those  modifications  affected 
the  output.  Major  Davies  then  took  the  additional  step  of  studying  time  savings  obtained 
by  running  the  simulation  on  parallel  processors. 

The  methodology  used  in  this  study  emulates  the  five-step  process  used  by  Major 
Davies  in  the  execution  of  his  study.  First,  input  variables  affecting  aspects  under  study 
were  determined.  In  the  case  of  this  effort,  those  parameters  in  THUNDER  determining 
effectiveness  of  air  packages  were  manipulated  (Davies,  1998:  46). 

Major  Davies  then  selected  output  metrics  to  compare  between  different 
scenarios.  These  included  exchange  ratios  (number  of  air-to-air  kills  for  Blue  [friendly] 
side  divided  by  those  of  the  Red  [enemy]  side),  total  amount  of  enemy  equipment 
destroyed,  and  days  needed  to  decrease  Red  air  strength  to  10  percent  of  its  original  level, 
defined  as  air  superiority.  After  determining  output  metrics,  the  experiment  was  then 
designed.  In  Major  Davies’  case,  this  included  modifying  the  appropriate  input  data  files 
and  planning  the  simulation  runs  such  that  a  minimum  number  or  runs  would  be  required 
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to  obtain  the  desired  results  the  first  time  through,  without  the  need  of  additional  runs  at  a 
later  time  (Davies,  1998;  70). 

Step  four  included  designing  the  parallel  system  in  which  to  run  the  simulations 
necessary  for  his  thesis.  This  was  included  as  a  sub-task  by  his  thesis  sponsors  as  a 
means  to  validate  the  concept  of  the  Simulation  and  Analysis  Facility  (SIMAF)  at  the 
Aeronautical  Systems  Center  and  will  not  be  conducted  in  this  effort.  Finally,  Major 
Davies  conducted  the  simulation  runs,  then  collected,  analyzed,  and  reported  the 
outcomes  (Davies,  1998;  73). 

The  output  metrics  chosen  by  Major  Davies  are  of  particular  interest  since  they 
suggest  useful,  validated  measurements  that  can  be  used  in  other  studies.  Major  Davies 
considered  five  measurements  of  interest.  The  air-to-air  exchange  ratio,  defined  as  Blue 
air-to-air  kills  divided  by  Red  air-to-air  kills  suggests  the  expected  relative  number  of 
kills  for  both  sides  during  a  protracted  engagement.  Surface  to  air  exchange  ratio 
provides  insight  into  the  vulnerability  of  each  side’s  aircraft  to  the  opposition’s  air 
defenses.  This  statistic  can  also  suggest  the  effectiveness  of  air  defenses  for  both  sides 
(Davies,  1998;  70). 

Total  number  of  Red  tanks,  artillery,  armored  personnel  carriers,  trucks,  and 
infantry  vehicles  destroyed  indicates  the  effectiveness  of  the  Blue  side  in  attacking  air-to- 
ground  targets.  Days  to  achieve  air  superiority  was  also  examined.  On  the  surface,  this 
measurement  would  seem  to  lend  insight  into  the  effectiveness  of  the  Blue  forces  in  air- 
to-air  combat.  Looking  deeper  into  this  statistic  can  also  reveal  characteristics  about  the 
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Red  forces.  Early  attrition  to  10  percent  of  day  one  forces  (the  definition  of  air 
superiority)  can  suggest  an  aggressive  Red  force  (Davies,  1998:  70). 

Finally,  the  war  on  the  ground  as  simulated  by  THUNDER  is  affected  by  the  air 
war.  Days  to  halt  PLOT  (forward  line  of  operating  troops,  that  is,  the  “front”)  movement 
is  also  a  useful  measure.  This  is  defined  as  the  number  of  days  which  elapse  from  the 
time  the  Blue  forces  begin  losing  ground  (usually  the  outset  of  a  repetition)  to  when  they 
halt  the  Red  advance  and  begin  reclaiming  ground  (Davies,  1998:  70). 

Lieutenant  Colonel  John  Siegner  also  used  THUNDER  in  the  accomplishment  of 
his  thesis.  Analysis  of  Alternatives:  Multivariate  Considerations.  Lt  Col  Siegner  used 
THUNDER  as  a  way  of  comparing  different  weapon  systems  being  considered  for 
acquisition.  Like  Major  Davies,  Lt  Col  Seigner  was  also  validating  the  SIMAF  at  the 
Aeronautical  Systems  Center,  which  proposed  to  run  analyses  of  alternatives  (AOAs)  for 
proposed  acquisitions.  Data  describing  a  notional  aircraft,  the  F-XX,  and  a  notional  air- 
to-air  missile,  the  AJM-XX,  were  input  into  THUNDER  and  compared  with  known  data 
from  the  F-15C  and  the  AIM-9  air  intercept  missile.  These  comparisons  served  as  a  way 
to  determine  those  performance  characteristics  which  would  be  sought  after  in  new 
acquisitions  (Siegner,  1998:  1-4). 

Lt  Col  Siegner  introduces  the  idea  of  data  mining  as  a  means  of  extracting 
information  from  large  sets  of  raw  data.  Data  mining  by  way  of  visualization  is  the  most 
elementary  method,  but  is  only  effective  for  small  data  sets  with  easily  distinguished 
elements.  More  sophisticated  software  packages  apply  analytical  tools  and  mathematical 
techniques  to  larger  data  sets  to  obtain  desired  information  (Siegner,  1998:  17-19). 
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Linking  Procurement  Dollars  to  an  Alternative  Force  Structures'  Combat 
Capability  Using  Response  Surface  Methodology  is  the  title  of  a  thesis  completed  by 
Major  James  Grier  that  used  THUNDER  to  simulate  combat  for  different  prospective 
force  structures.  The  objective  was  to  compare  different  force  mixes  of  weapon  systems 
in  terms  of  combat  effectiveness  and  total  force  cost.  This  information  is  useful  input  for 
the  planning,  programming,  and  budgeting  system  (PPBS)  used  by  the  Federal 
Government  to  determine  those  projects  which  receive  federal  funding  and  those  that  do 
not.  In  this  manner.  Major  Grier  used  THUNDER  as  a  means  to  determine  the 
effectiveness  of  force  packages  under  consideration  for  federal  funding.  Those  force 
mixes  which  provide  optimal  combinations  of  combat  effectiveness,  cost,  and  other 
constraints  can  be  used  to  influence  planning  during  the  budgeting  process  (Grier,  1997: 
1-3). 

THUNDER  provided  seven  output  parameters  of  interest  on  which  Major  Grier 
performed  a  stepwise  regression  analysis  to  identify  those  weapon  systems  which 
significantly  affected  overall  system  capability.  Major  Grier  also  performed  a  second- 
order  analysis  to  determine  interaction  between  variables.  Systems  which  were  found  to 
impact  output  were  used  to  build  an  1 1 -space  surface  (the  response  surface  used  in  the 
response  surface  methodology).  The  surface  was  defined  as  the  set  of  points  resulting 
from  all  possible  values  of  independent  variables  entered  into  the  regression  equation. 

The  regression  equation  was  used  as  an  objective  function  in  an  integer  programming  (IP) 
problem  to  determine  the  most  effective  force  mix  given  a  set  of  constraints  (Grier,  1997: 
60). 
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Constraints  included  procurement  dollars,  operations  and  maintenance  costs, 
production  limitations,  treaty  limits,  and  integer  values  for  decision  variables.  The 
solution  of  the  resulting  IP  is  the  most  effective  weapon  system  mix  of  aircraft  and 
munitions  given  the  aforementioned  constraints  (Grier,  1997:60).  The  thesis  was 
evidence  of  THUNDER'S  versatility  beyond  simple  campaign  analysis.  Major  Grier 
looked  “beyond  the  war"  and  analyzed  the  big  picture  of  military  war  planning. 

Logistics  Research 

Since  this  thesis  deals  also  with  the  logistics  involved  with  deploying  aircraft  and 
support  equipment,  past  research  in  these  areas  was  also  of  interest.  Development  and 
Analysis  of  a  Dual-Role  Fighter  Deployment  Footprint  Logistics  Planning  Equation  by 
Captain  Stanley  Griffiss  and  Captain  Joseph  Martin  consisted  of  the  development  of 
linear  regression  equations  for  which  the  number  of  primary  assigned  aircraft  (PAA) 
served  as  the  independent  variable.  These  regression  models  estimated  the  number  of  C- 
141  equivalents  required  to  support  the  Joint  Strike  Fighter  (JSF)  for  a  period  of  thirty 
days  (Griffis  and  Martin,  1996:  31). 

Their  research  was  of  particular  interest  because  it  could  provide  an 
approximation  for  the  logistics  footprints  of  the  aircraft  studied  in  a  thesis  effort  such  as 
this.  Griffis  and  Martin  obtained  26  unit  type  codes  (UTCs;  documents  used  for  logistics 
planning  purposes)  for  various  numbers  of  aircraft  (primary  assigned  aircraft,  or  PAA)  of 
the  Block  30  F-16,  Block  40  F-16,  Block  50  F-16,  F-15E,  F-1 17A,  and  A-10.  From  each 
of  the  UTCs,  they  extracted  the  number  of  short  tons  of  equipment  needed  to  support  the 
given  package.  They  chose  such  a  wide  variety  of  aircraft  because  no  deployment  data 
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currently  exists  for  the  JSF  (the  JSF  is  scheduled  for  fielding  in  2007)  and  it  was  expected 
that  data  would  average  out  over  all  MDSs  included  in  the  study  (Griffis  and  Martin, 

1996:  45). 

Logistics  data  for  this  effort  were  supplied  by  personnel  in  the  Logistics  Plans 
office  at  HQ  ACC.  However,  the  Griffis  and  Martin  thesis  could  have  been  used  in  the 
absence  of  viable  support  data.  If  their  work  is  to  be  used  in  this  manner,  the  user  should 
carefully  examine  its  applicability.  There  are  two  disconnects  between  the  work  of 
Griffis  and  Martin  and  this  thesis  effort.  The  UTCs  used  in  the  JSF  study  were  intended 
for  use  in  a  30-day  deployment.  This  effort  concentrates  on  a  seven  day  deployment,  the 
expected  length  of  time  an  AEF  would  be  active.  (If  an  AEF  operated  in  a  situation  that 
was  expected  to  exceed  that  amount  of  time,  a  longer  deployment  package  consisting  of  a 
more  diverse  air  package  would  be  utilized.)  Although  exact  deployment  numbers 
specifically  describing  an  AEF  do  not  currently  exist,  a  researcher  could  assume  that  the 
relationship  between  the  numbers  obtained  by  Griffis  and  Martin  will  maintain  their 
relationship  when  interpolated  to  a  seven-day  deployment.  That  is,  if  the  amount  of 
equipment  required  to  support  twelve  F-15Es  for  thirty  days  is  25  percent  more  than  the 
amount  of  equipment  required  to  support  twelve  Block  50  F-16s  for  thirty  days,  then  the 
amount  of  equipment  required  to  support  twelve  F-15Es  and  twelve  Block  50  F-16s  for 
seven  days  will  hold  approximately  the  same  relationship. 

The  other  difference  between  the  two  studies  is  that  F-15Cs  are  included  in  this 
effort.  Griffis  and  Martin  did  not  use  any  F-15C  UTCs  in  their  analysis.  The  regression 
equations  derived  in  the  JSF  study  are  “.  ..proof  of  coneept  for  modeling  any  future 
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weapon  system’s  deployment  footprint. .  .’’(Griffis  and  Martin,  1997;  vii).  Since  they 
were  created  using  data  from  existing  fighter  aircraft,  a  researcher  could  assume  that  the 
regression  models  would  suggest  deployment  data  for  the  F-15C  with  as  much  statistical 
confidence  (if  not  more  confidence,  due  to  the  fact  that  the  input  data  represent  current 
aircraft  and  current  support  methods). 

Logistics  data  used  in  this  analysis  came  from  the  Logistics  Plans  office  at  the 
AEF  Battlelab  and  the  Logistics  Plans  Office  at  Headquarters,  Air  Combat  Command. 
The  data  consist  of  approximations  of  the  number  of  short  tons  required  for  a  given  MDS. 
Since  there  have  currently  been  no  rapid  response  AEF  deployments  to  date,  only 
approximations  exist  for  support  required.  The  data  used  in  this  study  is  discussed  in 
depth  in  the  Findings  and  Analysis  chapter. 


26 


III.  Methodology 


This  chapter  describes  the  methodology  employed  during  the  execution  of  this 
study.  It  begins  with  an  explanation  of  THUNDER,  the  simulation  package  used  to 
produce  data  on  combat  effectiveness  which  will  serve  as  a  basis  of  comparison  between 
the  five  air  packages.  This  chapter  then  describes  the  THUNDER  input  files  which  have 
a  significant  impact  on  this  study.  Information  in  these  files  defines  the  particular 
scenario-package  combination  under  investigation. 

These  files  contain  information  such  as  probability  of  kill  (PK)  data,  aircraft 
performance  and  maintenance  data,  command  and  control  information,  satellite 
capabilities,  and  surveillance  and  reconnaissance  data.  This  base  scenario  is  copied  to 
other  files  for  use  in  future  runs.  Information  in  these  files  which  does  not  remain 
constant  includes  the  number  and  type  of  aircraft  and  enemy  air  defenses.  These 
deviations  from  the  Storm  scenario  define  the  particular  air  package  and  scenario  used  in 
each  combination. 

The  section  explaining  the  design  of  the  experiment  describes  the  interaction 
between  air  packages  and  scenarios.  A  description  of  each  scenario  that  is  used  as  the 
basis  of  comparison  between  the  five  packages  is  given  in  order  to  expose  the  relative 
strengths  and  weaknesses  of  each  air  package.  Information  such  as  number  of  enemy 
aircraft  and  surface  to  air  threat  defines  each  scenario  and  is  held  constant  while  output 
data  from  the  five  air  packages  are  compiled. 

For  each  scenario  in  the  base  case,  the  individual  air  packages  are  run  through  the 
simulation  and  the  resulting  output  analyzed  for  comparison.  The  bases  of  comparison 
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between  the  five  packages  are  termed  measures  of  effectiveness  (MOEs).  Criteria  for 
selecting  MOEs  and  their  significance  to  the  study  are  discussed,  as  well  as  how  the  data 
were  extracted  from  TEnJNDER  output.  The  chapter  then  explains  the  procedure  used  to 
obtain  the  rank  order  of  effectiveness  of  the  air  packages  for  each  of  the  MOEs. 

After  the  procedure  leading  to  a  rank  order  of  the  effectiveness  for  all  scenario-air 
package  combinations  is  reviewed,  an  explanation  how  the  supportability  requirement  for 
each  package  is  given.  Data  for  support  equipment  requirements  were  supplied  by 
personnel  in  Logistics  Plans  at  HQ  ACC.  This  requirement,  expressed  in  terms  of  short- 
tons  of  equipment,  is  important  to  this  study  because  it  Will  be  a  limiting  factor  when  the 
decision  is  made  to  deploy  a  force.  The  discussion  includes  the  source  of  the  numbers, 
assumptions  which  are  made  when  using  the  figures,  and  other  factors  of  concern  to  the 
logistics  analyst. 

THUNDER  Input  Data  Files 

squadron.dat  -  The  first  section  of  the  squadron.dat  file  assigns  a  four-digit 
number  to  each  type  of  aircraft  in  theater  and  defines  sortie  profiles.  The  four-digit 
identification  assigned  to  each  MDS  is  used  in  other  files  to  reference  a  particular  type  of 
aircraft.  The  profiles  describe  the  number  of  sorties  each  aircraft  can  fly  on  a  given  day 
of  the  war,  in  terms  of  sustained  generation  (AUTH.QTY.  SORT/DAY)  and  a  surge 
capability  (AC. MAX. SORT/DAY)  numbers  and  maximum  capability  (THUNDER  Analyst 
Manual  [TAM],  Version  6.4,  Vol  3;  11:  1996). 

The  second  section  of  the  file  defines  each  individual  squadron  in  theater. 

SUP.  CMD.ID  is  a  four-digit  number  assigned  in  the  airplan.dat  file  which  defines 
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capabilities  of  squadrons  with  a  particular  command  identification.  TYPE.AC.ID  is  the 
four-digit  number  assigned  to  each  MDS  in  the  first  section  of  this  file.  A  UTH.  QTY 
defines  the  number  of  aircraft  in  theater  from  that  a  particular  squadron.  This  is  the 
number  that  is  manipulated  to  represent  the  air  packages  as  defined  earlier.  SERV.KIT.ID 
is  the  four-digit  number  assigned  to  the  service  kit  for  each  MDS  in  the  acserv.dat  file 
(TAM,  Version  6.4,  Vol  3;  1996:  1 1). 

Finally,  for  each  squadron,  the  strength  of  that  squadron  in  performing  each 
mission  in  THUNDER  is  defined.  These  are  scaled  from  zero  (no  effectiveness)  to  100 
(perfectly  effective).  These  numbers  are  multiplied  by  similarly  defined  factors  in  the 
typeac.dat  file  to  determine  the  squadron’s  effectiveness  at  performing  a  given  mission. 
(The  typeac.dat  file  defines  each  MDS’s  effectiveness,  the  squadron.dat  file  defines  each 
squadron’s  effectiveness.)  The  product  is  used  by  THUNDER  to  determine  the  overall 
effectiveness  in  the  routine  used  to  assign  squadrons  to  missions.  It  is  in  this  manner  that 
squadrons  are  created  to  perform  the  three  missions  of  air-to-air,  air-to-ground,  and 
SEAD  (TAM,  Version  6.4,  Vol  3;  1996:  1 1). 

airmuntdat  -  The  airmunt.dat  file  assigns  each  of  the  munitions  programmed  in 
THUNDER  a  three-digit  number  identifying  that  munition  in  other  files.  The  file  also 
lists  the  function,  weight,  effective  radius,  and  other  information  about  each  munition. 
The  harmpkdat  file  lists  those  mimitions  used  to  suppress  enemy  radar  site  by 
identification  number  as  assigned  in  airmunt.dat  (TAM,  Version  6.4,  Vol  3;  1996: 10). 

harmpk.dat  -  The  harmpkdat  file  contains  a  section  entitled  RADAR. KILLER. 
IDS.  Within  this  section,  BLUE..ID  lists  the  munition-aircraft  combination  for  the  AEF. 
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For  the  purpose  of  this  study,  the  only  combination  listed  is  the  Block  50  F-1 6  deploying 
the  AGM-88.  The  AGM-88  is  also  known  as  the  Mgh-speed  ^ntiiJadiation  Mssile 
{HARM).  The  aircraft  (Block  50  F-16)  is  identified  by  the  four-digit  designation 
assigned  in  the  squadron.dat  file,  harmpkdat  uses  the  three-digit  number  used  in 
airmunt.dat  to  identify  each  weapon  and  assigns  a  five-digit  number  to  define  possible 
HARM/aircraft  combinations  and  their  PKs  (TAM,  Version  6.4,  Vol  3;  1996:  9). 

The  next  section  in  harmpkdat  is  entitled  HARM.SPW.  VS.ENEMY.GROUND. 
RADAR.PKS.BY.NUM.AD.  TARGETS.BAND.  This  section  assigns  the  PK  for  each  five¬ 
digit  aircraft/antiradiation  missile  combination  against  enemy  air  defense  sites.  The 
coordinates  for  enemy  air  defense  locations  are  given  and  each  assigned  a  four-digit 
number  in  adcomplex.dat.  This  number  is  how  each  site  is  referred  to  in  harmpkdat 
(TAM,  Version  6.4,  Vol  3;  1996:  9). 

The  PK  values  in  harmpkdat  are  assigned  based  on  a  function  vdth  independent 
variable  values  from  zero  to  one.  PKs  are  dependent  variables  defined  at  interpolated 
points  on  the  function.  The  PK  for  the  AGM-88  iifRM  missile  is  defined  as  0.40  and 
held  constant  throughout  all  packages  (TAM,  Version  6.4,  Vol  3;  1996:  9). 

acmaintdat  -  The  acmaint.dat  file  defines  maintenance  characteristics  of  aircraft 
involved  in  the  simulation.  The  maximum  time  for  each  side  to  return  a  damaged  aircraft 
to  operation  is  defined,  as  well  as  three  degrade  curves.  The  degrade  curves  produce  a 
factor  by  which  baseline  maintenance  times  are  multiplied  based  on  the  percentage  of  an 
airbase’s  maintenance  capability  that  is  available  at  the  time  an  aircraft  needs  repair.  The 
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file  defines  a  degrade  curve  for  each  of  three  types  of  maintenance  activities: 
rearm/refuel,  short  term  repair  and,  long  term  repair  (TAM,  Version  6.4  Vol  3;  1996: 1 1). 

Aircraft  are  differentiated  by  their  four-digit  identifier  assigned  in  the 
squadron.dat  file.  Under  each  aircraft  type,  the  user  decides  whether  or  not  high  or  low 
resolution  maintenance  data  are  desired.  High  resolution  takes  aircraft  battle  damage 
from  each  possible  mission  profile  into  accoimt  and  what  types  of  damage  are  possible. 
Low  resolution  simply  assigns  an  aggregate  probability  of  short  term  and  long  term 
maintenance  needs.  Regardless,  rearm  and  refuel  time  distributions  are  defined,  as  well 
as  which  of  the  degrade  curves  defined  above  will  be  used  to  produce  the  repair  time 
multiplier  (TAM,  Version  6.4,  Vol  3;  1996: 1 1). 

If  low  resolution  maintenance  data  are  desired  by  the  user,  they  are  produced 
based  on  the  user-defined  probabilities,  and  repair  time  distributions  are  defined  (for  both 
short  and  long  term  maintenance  performed  at  either  the  primary  [main]  base  or  a 
secondary  [divert]  base)  given  that  an  aircraft  sustained  battle  damage  (TAM,  Version 
6.4,  Vol  3;  1996:  11). 

If  high  resolution  maintenance  data  are  required,  probabilities  of  sustaining  battle 
damage  are  defined  by  the  user  for  each  of  the  possible  mission  profiles.  Additionally, 
probabilities  of  sustaining  various  different  types  of  battle  damage  (sheet  metal,  avionics, 
structures,  hydraulics,  etc)  are  defined  and  used  to  determine,  for  instance,  the  probability 
of  requiring  long  term  maintenance  on  a  rudder.  Repair  time  distributions  are  also 
defined  based  on  whether  or  not  the  repairs  are  conducted  at  the  main  base  or  an  alternate 
base  (TAM,  Version  6.4,  Vol  3;  1996:  1 1). 
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typeac.dat  -  The  typeac.dat  file  defines  characteristics  for  each  MDS  in  the 
theater.  The  radar  cross  section  (RCS)  is  given  for  each  aircraft,  as  well  as  the  available 
defensive  air-to-air  loads.  Aircraft  are  again  identified  by  the  four-digit  numbers  assigned 
in  the  squadron,  dat  file.  For  each  aircraft,  performance  characteristics  of  seven  flight 
profiles  are  defined  in  terms  of  altitude  and  speed.  Length  of  runway  required  for  take  off 
and  landing  are  given  for  various  situations.  The  number  of  sorties  an  aircraft  is  capable 
of  flying  during  day  and  night  operations  are  given  (TAM,  Version  6.4,  Vol  3;  1996: 

10). 

Mission  data  are  also  defined.  This  includes  the  minimum  flight  size,  dimensions 
of  combat  patrol  box,  and  the  number  of  targets  that  can  be  attacked  per  sortie.  Of 
primary  concern  in  this  effort  are  the  configuration  data.  Additionally,  effectiveness  for 
each  mission  performed  is  defined  in  this  section.  These  values  are  used  along  with 
similar  values  in  the  squadron,  dat  file  to  determine  the  effectiveness  of  each  squadron 
performing  a  given  mission.  Altitude  (high  or  low)  is  also  defined  for  each  mission 
(TAM,  Version  6.4,  Vol  3;  1996:  10). 

The  next  section  in  the  file  defines  fuel  configurations  and  assigns  a  four-digit 
number  to  each.  The  configurations  define  capacity,  amount  of  first  and  second  refuels 
and  the  increase  in  RCS,  if  any.  Bum  profiles  (rates  in  pounds  per  minute)  are  defined  in 
this  section  for  each  flight  profile  and  also  assigned  a  four-digit  number  for  use  later  in 
the  file  (TAM,  Version  6.4,  Vol  3;  1996:  10). 

The  next  section  of  the  file  defines  weapons  configurations  available  for  both  air- 
to-air  and  air-to-ground  missions.  The  section  contains  the  types  of  ordnance  carried  and 
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the  quantities  of  each.  Fuel  configuration  and  bum  profiles  are  included  for  each 
configuration  as  defined  above,  as  well  as  the  increase  in  RCS  of  the  aircraft/mimitions 
combination  attributable  to  the  load.  For  each  configuration,  launches  per  air-to-air 
engagement  are  given,  as  well  as  the  percent  of  available  command  and  control  with  and 
without  advanced  early  warning  (AEW)  aircraft  (TAM,  Version  6.4,  Vol  3;  1996: 1 1). 

Mission  effectiveness  for  each  configuration  can  be  entered  to  affect  the  overall 
effectiveness  as  defined  in  the  squadron,  dat  and  in  the  aforementioned  section  of 
typeac.dat.  Information  about  each  type  of  munition  is  given  based  on  the  three-digit 
number  assigned  in  airmunt.dat  in  terms  of  quantity,  circular  error  probability  (CEP),  and 
the  increase  in  radar  configuration  attributable  to  a  particular  munition.  Finally,  radar 
jamming  capability  is  defined  (TAM,  Version  6.4,  Vol  3;  1996:  1 1). 

airplan.dat  -  The  airplan.dat  file  defines  the  air  war.  For  each  of  nineteen 
missions,  the  need  for  air  escort  is  determined  and  the  importance  of  escort  for  each 
mission  is  entered.  Additionally,  the  need  for  radar  jamming  and  SEAD  is  determined 
and  importance  of  such  support  is  scaled  from  1  to  100  (TAM,  Version  6.4,  Vol  3;  1996: 
15). 

The  file  also  defines  characteristics  of  commands  for  use  in  other  files  and  assigns 
each  a  four-digit  number.  Characteristics  include  time  over  target  spacing  in  minutes  and 
the  percent  of  aircraft  needed  to  continue  a  given  mission.  The  primary  and  secondary 
missions  for  each  command  are  defined.  After  assigning  these  attributes  for  each 
command  on  each  side,  the  file  takes  the  next  step  and  defines  which  commands  on  each 
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side  can  perfonn  each  of  the  missions  which  THUNDER  simulates  (TAM,  Version  6.4, 
Vol3;  1996:  15). 

airalloc.dat  -  The  airalloc.dat  file  defines,  for  each  command,  the  relative 
effectiveness  of  the  command  in  performing  a  particular  mission  with  respect  to  other 
missions  in  a  particular  mission  class.  For  instance,  included  in  the  air  superiority  class 
are  BARCAP  (barrier  combat  air  patrol),  AIRESC  (air  escort),  and  FSWP  (fighter 
sweep).  A  number  representing  the  effectiveness  of  a  command  at  performing  each  of 
these  missions  is  assigned  in  the  file.  The  effectiveness  numbers  assigned  must  sum  to 
1 00  for  each  mission  class.  The  command  identifiers  and  characteristics  as  defined  in 
airalloc.dat  are  used  in  squdron.dat  to  further  define  capabilities  of  each  squadron  (TAM, 
Version  6.4,  Vol  3;  1996:  14). 

adcompbadat  -  This  file  simply  lists  air  defense  sites  for  each  side.  Included  are 
location,  strength,  and  munitions  at  each  site  (TAM,  Version  6.4,  Vol  3;  1996:  1 1). 
Experiment  Design 

This  study  compares  the  relative  combat  effectiveness  of  five  different  air 
packages  as  defined  by  the  AEF  Battlelab.  A  second  scenario  was  created  in  an  attempt 
to  find  a  situation  which  changed  the  rank-order  of  effectiveness  of  each  package  from 
the  original  Storm  scenario.  A  scenario  which  changes  the  order  of  effectiveness  results 
in  the  conclusion  that  effectiveness  is  scenario-dependent.  The  significance  of  this 
finding  would  be  that  war  planners  must  consider  the  specific  environment  in  which  an 
AEF  would  operate  and  tailor  the  package  accordingly.  This  includes  considering  the 
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strengths  and  weaknesses  of  each  package  to  determine  an  appropriate  force  mix  for  a 
given  situation. 

The  term  experiment  will  be  used  to  describe  each  air  package-scenario 
combination.  With  five  air  packages  and  two  scenarios,  there  will  be  a  total  of  ten 
experiments.  Each  experiment  will  be  replicated  thirty  times,  for  a  total  of  300 
replications.  Each  replication  will  produce  one  data  point  for  each  of  six  MOEs  obtained 
from  THUNDER  output,  for  a  total  of  1 800  data  points.  MOEs  to  be  examined  are  as 
follows: 

1.  Number  of  Blue  air-to-air  losses  (BAAL) 

2.  Number  of  Red  air-to-air  losses  (RAAL) 

3.  Number  of  Blue  losses  to  Red  air  defenses  (BSAML) 

4.  PLOT  movement  (PLOT) 

5.  Number  of  Blue  air-to-ground  kills  (BACK) 

6.  Number  of  unscheduled  maintenance  activities  performed  on  Blue  aircraft 
(BMX) 

Scenario  One  -  The  first  scenario  is  the  original  Storm  scenario,  with  a  reduced 
enemy  air  presence  and  air  defense  system.  Original  numbers  of  AEW  control  and 
surveillance  aircraft  were  maintained  for  both  sides  (12  AW  ACS  and  6  JSTARS  aircraft 
for  the  AEP,  and  6  Mainstay  [Soviet  AEW  aircraft]  for  enemy  forces).  The  number  of 
B-52s  in  theater  was  also  set  constant  at  six.  This  research  assumes  that  these  aircraft  are 
a  “fixed  cost”  of  conducting  tin  air  war  and  would  be  included  in  any  package  either  side 
would  send  to  the  fight. 

When  only  the  AEP  aircraft  were  entered  into  squadron.dat  (and  enemy  forces 
left  intact)  in  trial  runs,  the  result  was  useless  data;  the  AEP  was  eliminated  early  in  the 
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scenario.  Obviously,  an  AEF  would  not  be  an  appropriate  response  for  a  scenario  such  as 

defined  by  Storm.  This  necessitated  the  modifications  to  the  Storm  scenario. 

Conversely,  eliminating  too  much  of  the  enemy  threat  results  in  an  unrealistic 

advantage  in  favor  of  the  AEF.  The  end  product  of  this  force  mix  is  also  useless  data. 

Care  was  taken  to  modify  the  numbers  of  enemy  forces  in  such  a  way  as  to  result  in 

meaningful  data.  The  final  result  was  that  a  force  consisting  of  36  MiG-23s  produced 

data  that  was  useful.  The  MiG-23  is  an  appropriate  choice  of  aircraft  to  engage  because  it 

represents  a  formidable  opponent  in  both  the  air-to-air  and  air-to-ground  role  (according 

to  the  original  squadron.dat  and  typeac.dat  files  in  the  Storm  scenario).  The  number  of 

MiG-23s  was  determined  by  simply  equaling  the  number  of  AEF  aircraft  (36). 

Although  this  fabricated  enemy  force  produced  useful  data,  a  homogeneous 

enemy  force  structure  is  not  realistic.  To  create  a  realistic  enemy  force  structure,  the 

1998  edition  of  Jane ’s  World  Air  Forces  (JWAF)  was  consulted.  According  to  JWAF, 

Iraq  maintains  an  inventory  of  60  MiG-23  s,  8  MiG-29s,  60  F-ls,  and  40  MiG-2  Is 

(Jackson,  1998).  If  there  are  36  enemy  aircraft  in  theater  in  the  same  proportion  as  the 

aircraft  above,  the  following  force  mix  is  produced: 

13  MiG-23s 
2  MiG-29s 
13  F-ls 
8  MiG-2  Is 

The  original  adcomplx.dat  file  contained  forty  enemy  air  defense  sites.  This 
number  was  then  doubled  in  an  effort  to  create  the  second  scenario.  This  modification 
resulted  in  a  substantial  number  of  air  defense  losses  for  AEF  aircraft.  If  the  second 
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scenario  was  to  have  twice  the  number  of  air  defense  sites,  the  number  must  be  reduced  in 

scenario  one.  The  number  was  therefore  reduced  to  twenty  to  define  the  first  scenario.  It 

is  the  above  mix  of  enemy  aircraft  (along  with  six  Mainstay  aircraft)  and  each  of  the  five 

AEF  packages  (each  AEF  package  also  including  twelve  AW  ACS  aircraft,  six  JSTARS 

aircraft,  and  six  B-52s)  which  constitute  the  first  five  experiments  in  this  study. 

Scenario  Two  -  The  second  scenario  restores  the  number  of  enemy  air  defense 

sites  to  its  original  value  of  forty  and  decreases  the  number  of  enemy  aircraft.  The  same 

process  was  followed  to  create  a  scenario  which  resulted  in  meaningful  data  as  was 

followed  to  create  the  first  scenario.  With  a  relatively  small  number  of  aircraft  originally 

in  theater,  there  could  be  little  decrease  in  the  original  number  to  result  in  output  worth 

investigating.  After  experimentation,  it  was  determined  that  reducing  enemy  aircraft  by 

25  percent  produced  consequential  data.  Keeping  original  proportions  intact,  this  resulted 

in  the  following  enemy  force  mix: 

10MiG-23s 
1  MiG-29 
lOF-ls 
6  MiG-2  Is 

This  new  force  mix  was  entered  into  the  squadron.dat  file  by  inputting  the  appropriate 
number  under  A  UTH.  QTY  for  each  aircraft. 

Random  Number  Generation  -  The  same  random  number  stream  is  used 
throughout  all  experiments  to  determine  the  outcome  of  engagements,  resulting  in 
correlated  data.  This  procedure  is  known  as  correlated  sampling.  Correlated  sampling 
is  utilized  due  to  the  fact  that  it  “. .  .usually  reduces  the  variance  of  the  estimated 
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difference  of  the  performance  measures  and  thus  can  provide,  for  a  given  sample  size, 
more  precise  estimates  of  the  mean  difference  than  can  independent  sampling.”  (Banks, 
Carlson,  and  Nelson;  1996:  475).  This  method  of  testing  reduces  the  impact  of  the 
random  numbers  themselves  on  the  outcomes,  and  result  in  actual  performance 
parameters  of  weapon  systems  having  a  greater  impact  on  decisions.  Any  bias  that  the 
random  numbers  may  introduce  into  the  system  is  dampened  by  the  same  numbers  being 
used  in  each  experiment. 

The  version  of  THUNDER  used  in  this  study  has  ten  built-in  random  number 
generating  streams.  Although  it  is  impossible  to  assign  each  function  its  own  random 
number  seed,  the  ten  which  are  available  were  assigned  to  blocks  of  functions.  However, 
according  to  Lt  Col  Seigner,  .  .the  complexity  and  multitude  of  interactions  in  the 
model  contribute  to  the  uncertainty  of  the  variance  reduction  achieved  through  the 
synchronization  scheme...”  (Seigner,  1998: 40-41).  Although  the  streams  will  still 
diverge  within  a  given  function,  this  is  the  advised  method  of  using  the  available  random 
number  streams  in  attempting  to  reduce  variance. 

Rank  Ordering  Effectiveness 

To  get  an  idea  of  the  rank  order  of  effectiveness,  the  five  air  packages  are  run  in 
THUNDER  in  the  base  scenario,  each  with  thirty  replications.  Statistical  tests  are 
conducted  on  each  of  the  output  metrics  to  determine  the  effectiveness  rankings.  The 
procedure  is  then  repeated  for  scenario  two  in  an  attempt  to  create  a  different  rank  order 
of  the  five  packages.  When  all  experiments  are  complete,  ten  tables  are  constructed  (one 
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for  each  scenario-package  combination)  with  a  column  heading  for  replication  number 
and  each  MOE.  These  tables  appear  in  Appendix  A. 

Bonferroni  Procedure  -  This  approach  allows  the  analyst  to  control  experiment- 
wise  error  when  selecting  the  “best”  package  for  each  MOE.  The  definition  of  best 
depends  on  the  particular  MOE  being  examined.  When  comparing  the  packages  in  a 
scenario  based  on  the  number  of  Red  air-to-air  losses,  the  best  is  determined  by  that 
package  which  scores  the  highest.  When  a  comparison  is  made  based  on  the  number  of 
Blue  air  defense  losses,  the  best  is  determined  by  that  package  which  scores  the  lowest. 

Since  each  MOE  is  compared  with  respect  to  five  air  packages,  there  are  a  total  of 
5(5-l)/2  =10  comparisons  which  will  be  made  for  each  MOE.  For  each  comparison,  a 
1  -  =  99  percent  confidence  interval  will  be  constructed,  where  j  represents  a  particular 

comparison  (/  =  1 ,2, . . .  1 0).  5,  will  be  the  statement  that  the  zth  comparison  contains  the 
difference  being  estimated.  For  example,  Sj  could  be  “Number  of  air-to-air  losses  for 
Blue  aircraft  for  package  one  minus  the  number  of  air-to-air  losses  for  Blue  aircraft  for 
package  two  is  contained  with  the  interval  (1.5 , 2.5).”  (Banks,  Carson,  and  Nelson;  1996: 
492). 

The  Bonferroni  inequality  draws  a  relationship  between  the  probability  that  all  ten 
statements  are  true  and  the  desired  confidence  as  follows: 

P(all  statements  S,  are  true,  /  =1,2,...  10)  >  1  -  c^  =  1  -  =  0.90 

where  =  0.10  is  the  overall  (experiment-wise)  error  probability.  The 

Bonferroni  inequality  is  equivalently  stated  as: 
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P(one  or  more  statements  is  false,  i  =1 ,2, ...  1 0)  <  ,  or  alternately 

P(one  or  more  of  the  ten  confidence  intervals  does  not  contain  the  difference 
being  estimated)  <  Og  (Banks,  Carson,  and  Nelson;  1996:  492). 

The  significance  of  these  inequalities  is  that  they  hold  even  when  the  common 
random  number  sampling  technique  is  used.  This  property  makes  the  Bonferroni 
approach  attractive  for  the  purpose  of  this  study  as  a  means  of  comparing  output  of  the 
five  air  packages  (Banks,  Carson,  and  Nelson;  1996: 492). 

The  first  step  in  conducting  a  Bonferroni  comparison  is  to  define  the  overall  error 
probability,  .  For  this  study,  ag=  0.10.  Next,  Oj  =  a^lO  =  0.01  for y=l,2,...10  since 
there  are  ten  comparisons  for  each  MOE  (Banks,  Carson,  and  Nelson;  492: 1996).  The 
thirty  replications  of  each  experiment  are  sufficient  to  assume  that  the  values  for  the 
differences  between  each  package  for  each  MOE  are  normally  distributed  according  to 
the  Central  Limit  Theorem,  but  the  sample  variance  (variance  of  the  differences)  still 
must  be  calculated  to  determine  statistical  significance  (McClave  and  Benson,  1994: 
282). 

Recall  that  there  are  10  comparisons  of  air  packages  for  each  MOE.  The 
comparison  between  each  two-package  combination  is  based  on  confidence  intervals  for 
the  difference  between  mean  values  of  MOEs.  For  example,  the  first  confidence  interval 
to  be  constructed  will  be  for  the  difference  in  Blue  air-to-air  losses  between  package  one 
and  package  two  in  scenario  one.  Letting  the  mean  of  the  thirty  replications  from 
package  one  equal  6j  and  the  mean  from  package  two  equal  $2,  the  confidence  interval 
will  be: 
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X  <6j-  02  ^  y 


where  x  is  the  lower  bound  and  is  the  upper  bound  of  the  interval.  If  is  greater  than 
zero  (and  assuming  x  <y),  then  the  quantity  6j  -  62  is  positive  within  the  specified 
confidence  and  therefore  6]  >  62-  If  y  is  less  than  zero  and  the  assumption  that  x  <y  is 
maintained,  the  quantity  6j  -  02  negative  within  the  specified  confidence  and  therefore 
01  <  02-  If  X  is  less  than  zero  and  y  is  greater  than  zero,  then  no  conclusions  can  be  drawn 
about  which  of  0/  and  02  is  greater,  and  therefore  no  ranking  will  exist  between  the  two 
air  packages  for  that  particular  MOE  (McClave  and  Benson,  1994:  869). 

The  lower  bound  on  the  interval,  x,  is  determined  by  the  following  formula: 

x  =  Mdu 

and  the  value  fory,  the  upper  boimd  on  the  confidence  interval,  is  determined  in  a  similar 
fashion: 


y  =  mdu  + 


where  is  the  mean  of  the  differences  0i  -  02  for  all  replications.  The  term  za^  is  the 
z-score  for  a  1  -  =  0.99  confidence  interval.  The  factor  ^  represents  the  standard 

deviation  of  the  differences  of  0j  -  02  for  all  replications,  and  n  is  the  number  of 
replications  (McClave  and  Benson,  1994:  869). 

Each  scenario-package  combination  has  ten  confidence  intervals  associated  with 
it,  for  a  total  of 

(5(5-l)/2)X6X  2  =  120 
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confidence  intervals.  Those  confidence  intervals  which  are  reported  appear  in  Appendix 
B.  Due  the  large  number  of  confidence  intervals  which  were  constructed,  only  those 
which  reveal  a  need  for  additional  replications  are  included. 

Logistics  Data 

Data  concerning  the  number  of  short-tons  required  to  support  each  package  were 
obtained  from  spreadsheets  and  current  UTCs  created  by  the  Air  Force  logistics  plans 
community.  Two  frequently  studied  air  packages  are  the  Eagle  Package  and  the  Falcon 
Package,  referred  to  as  package  one  and  package  three  respectively  in  this  study. 
Equipment  requirements  for  these  packages  are  broken  down  to  the  unit  level  by  item, 
allowing  the  analyst  to  determine,  for  example,  how  many  AM32A-86Ds  (power  units) 
are  needed  by  the  Block  50  F-1 6  squadron.  In  addition  to  this  information,  a  unit  weight 
and  total  weight  for  each  line  item  are  given,  allowing  the  researcher  to  determine  the 
total  weight  (in  short-tons)  of  equipment  needed  by  each  xmit. 

For  the  purpose  of  this  study,  only  equipment  used  by  aircraft  support  personnel 
(maintenance,  munitions  loaders,  etc.)  was  considered  in  the  evaluation.  Equipment  used 
by  personnel  in  fields  such  as  security,  services,  medical,  and  civil  engineering  were  not 
included.  (Such  equipment  would  also  likely  be  considered  a  “fixed  cost”  of  a 
deployment  and  would  not  experience  much  variance  from  package  to  package). 

Some  support  equipment  assumptions  are  necessary,  given  the  lack  of  data 
available  for  analysis.  No  true  AEF  has  occurred  to  date,  and  therefore  there  are  no 
validated  historical  data  which  can  be  used  as  basis  of  comparison  needed  for  this  study. 
The  spreadsheets  containing  the  data  needed  for  this  effort  were  supplied  by  the  Logistics 
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Plans  Office  at  Headquarters,  Air  Combat  Command  (ACC/LGX)  and  are  the  most  viable 
currently  available. 

Another  need  for  the  support  equipment  assumptions  mentioned  above  is  because 
only  four  of  the  five  aircraft  included  in  this  study  are  represented  in  the  Eagle  and 
Falcon  packages.  For  the  aircraft  not  included  in  the  packages  above,  the  Block  30  F-16, 
other  sources  must  be  sought  to  determine  supportability  requirements.  The  Griffis  and 
Martin  thesis  lists  validated  UTCs,  currently  in  use  by  the  Air  Force,  which  served  as 
inputs  for  the  development  of  the  regression  models  used  to  estimate  equipment 
requirements  for  the  JSF.  One  of  the  UTCs  listed,  3FKP10,  lists  support  required  for  a 
deployment  of  12  PAA  Block  30  F-16s  at  141 .2  short-tons.  This  particular  UTC  is 
known  as  an  aviation  UTC  and  includes  only  those  items  used  by  aircraft  support 
personnel. 

A  similar  restriction  was  in  effect  when  using  the  spreadsheets  supplied  by 
ACC/LGX.  Although  the  data  are  available  for  four  of  the  five  aircraft  used  in  this  study, 
attempting  to  estimate  the  total  support  required  for  each  package,  when  a  sole  source  for 
all  aircraft  does  not  currently  exist,  would  have  needlessly  complicated  the  issue.  More 
importantly,  the  resulting  data  would  experience  a  larger  margin  of  error  due  to  the 
nonavailability  of  a  sole  source  for  the  information  for  the  Block  30  F-16. 

The  numbers  for  the  first  four  packages  are  based  on  packages  currently  under 
study  by  the  Logistics  Plans  commimity.  Package  five  is  not  currently  being  considered 
as  a  viable  AEF  option,  so  there  are  no  data  on  which  to  base  an  approximation  for 
required  support.  The  Griffis  and  Martin  thesis  contains  a  list  of  several  current  Air 
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Force  UTCs  for  various  aircraft  and  PAAs.  It  is  on  these  18  PAA  F-16  UTCs  as  shown 
in  Table  1  (Griffis  and  Martin,  1996:  45)  that  an  approximation  for  required  support  for 
package  five  will  be  based. 


Table  1.  Package  Five  Logistics  Data 


Squadron 

Short-tons 

UTC 

Block  30 

225.9 

3FKM70 

Block  40 

272.4 

3FKM30 

Block  50 

213.2 

3FKAA0 

Total 

711.5 

The  following  chapter  contains  the  data  obtained  from  the  simulation  runs,  as  well 
as  the  results  of  the  Bonfeironi  method  of  rank  ordering  each  MOE.  The  chapter  also 
discusses  logistics  data  related  to  each  package  in  an  effort  to  obtain  knowledge  about  the 
relative  difficulty  in  deploying  each  package. 


44 


IV.  Findings  and  Analysis 


This  chapter  discusses  the  results  and  data  analysis  from  each  of  the  THUNDER 
replications  of  the  two  notional  scenarios  described  in  Chapter  3.  It  is  separated  by 
headings  describing  the  particular  MOE  under  investigation.  The  final  result  was  a  rank- 
order  of  each  of  the  packages  for  each  of  the  seven  measures,  where  justified  by  the 
described  statistical  procedures. 

The  scenario-package  combinations  are  referred  to  by  a  code  which  first  defines 
the  scenario,  then  the  air  package.  In  such  a  manner,  the  term  SnPm  defines  the  scenario- 
package  combination  of  package  m{m-  1,2,3,4,5)  flying  in  scenario  n  (n  =  1,2)  as  each 
were  defined  earlier. 

After  each  SiiPm  was  replicated  thirty  times,  the  data  were  analyzed  to  create 
tables  which  reflect  the  value  of  each  MOE  for  each  replication.  These  tables  appear  in 
Appendix  A.  Appendix  B  consists  of  tables  reflecting  comparisons  between  individual 
scenario-package  combinations  for  those  combinations  which  required  additional 
replications. 

Bonferroni  Method  of  Comparison 

The  Statistix  Analytical  Software  package  was  used  to  determine  the  rank-order 
of  the  packages  via  the  Bonferroni  procedure.  This  process  uses  the  mean  and  variance 
of  the  differences  of  all  two-member  combinations  of  treatment  means  (McClave  and 
Benson),  in  this  case,  air  packages.  The  data  in  Appendix  B  were  calculated  to  determine 
the  number  of  additional  runs  needed  for  the  procedure  to  be  more  decisive.  Those 
combinations  which  were  found  to  be  inconclusive  were  considered  for  additional  runs. 
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Additional  replications  were  accomplished  if  all  three  of  the  following  conditions  held: 

-  The  variance  of  the  differences  was  greater  than  10  percent  of  the  mean  of  the 

differences, 

-  The  99  percent  confidence  interval  contained  zero,  and 

-  If  it  could  be  shown  that  based  on  the  variance  of  differences  from  the  first  thirty 

runs,  zero  could  be  excluded  from  the  confidence  interval  by  including 
less  than  thirty  additional  runs. 

The  significance  of  the  99  percent  confidence  interval  is  that  this  level  precision  is 
required  to  assure  an  overall  confidence  of  90  percent  after  10  mean  comparisons.  Those 
differences  described  by  confidence  intervals  which  contained  zero  and  would  require 
more  than  thirty  additional  runs  to  exclude  zero  were  considered  to  be  too  small  to  be 
distinguished  by  THUNDER.  The  limit  for  additional  runs  was  placed  at  thirty  because  it 
served  as  a  practical  ceiling.  Some  comparisons  would  require  several  hundred  or  even 
thousands  of  additional  runs  to  conclude  that  a  difference  exists.  In  such  cases,  the 
conclusion  was  drawn  that  no  difference  could  be  observed  with  99  percent  confidence  by 
running  the  packages  through  THUNDER.  The  results  for  cases  which  called  for 
additional  runs  based  on  the  above  criteria  appear  in  Appendix  B. 

Rank-Order  of  MOEs 

A  review  of  information  significant  to  this  overall  study  will  refamiliarize  the 
reader  with  that  which  was  being  investigated.  The  air  packages  were  as  follows: 

Package  1 :  12  ea  F-1 5C  (Air-to-air) 

12  ea  F-15E  (Air-to-ground) 

12  ea  Block  50  F-16(SEAD) 

Package  2:  12  ea  Block  30  F-1 6  (Air-to-air) 

12  ea  F-15E  (Air-to-ground) 

12  ea  Block  50  F-16(SEAD) 
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Packages:  12  ea  F-15C  (Air-to-air) 

12  ea  Block  40  F-16  (Air-to-ground) 
12  ea  Block  50  F-16(SEAD) 

Package  4:  12  ea  Block  30  F-16  (Air-to-air) 

12  ea  Block  40  F-16  (Air-to-ground) 
12  ea  Block  50  F-16(SEAD) 

Package  5 :  1 8  ea  Block  30  F- 1 6  (Air-to-air) 

18  ea  Block  40  F-16  (Air-to-ground) 
18  ea  Block  50  F-16  (SEAD) 


Scenario  two  differs  from  scenario  one  in  that  scenario  2  has  twice  as  many 
enemy  air  defense  sites  (40)  and  25  percent  fewer  enemy  intercept  aircraft  (27)  as 
scenario  one.  Scenario  two  was  created  in  an  attempt  to  find  a  scenario  in  which  at  least 
two  air  packages  had  a  different  ranking  for  at  least  one  MOE,  thus  permitting  the 
conclusion  that  air  package  effectiveness  is  scenario-dependent. 

Blue  Air-to-air  Losses 

Table  2  below  summarizes  the  Blue  air-to-air  losses  data  for  both  scenarios. 


Table  2.  Blue  Air-to-air  Losses 
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This  measure  was  included  because  it  shows  how  vulnerable  the  air  package  is 


against  enemy  forces  in  the  air.  Table  2  is  typical  of  how  results  are  presented  for 


MOEs.  The  crosses  in  the  table  identify  the  group  or  groups  of  which  a  particular 


package  is  a  member.  The  columns  represent  homogeneity  groups.  No  statistical 
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difference  exists  between  members  of  the  same  group,  or  equivalently,  between  packages 
which  have  crosses  in  the  same  column.  In  scenario  one,  packages  one  and  three  were 
determined  to  lose  a  higher  number  of  aircraft  than  the  other  three  packages,  but  no 
statistical  difference  exists  between  those  two  packages,  as  evidenced  by  the  cross  for 
each  package  being  in  the  same  column.  Packages  two,  four,  and  five  were  in  the  next 
group  with  somewhat  fewer  losses.  Because  these  three  packages  are  in  the  same  group, 
as  indicated  by  the  crosses  for  each  package  being  in  the  same  column,  no  conclusions 
can  be  drawn  about  the  difference  in  losses  of  packages  two,  four,  and  five. 

After  the  first  thirty  replications,  it  was  determined  that  an  additional  fourteen 
replications  would  be  needed  for  a  difference  between  packages  two  and  five  to  become 
evident.  After  running  an  additional  thirty  replications,  a  decrease  in  the  absolute  value 
of  the  mean  and  a  slight  (12  percent)  increase  in  the  variance  of  the  differences  occurred, 
but  no  significant  statistical  difference  was  found  between  packages  two  and  five. 

Scenario  two  also  had  packages  one  and  three  in  the  group  with  the  most  losses. 
Packages  two,  four  and  five  made  up  the  other  group  in  which  there  was  no  statistical 
difference  between  means. 

Red  Air-to-air  Losses 

Red  air-to-air  losses  (RAAL)  were  measured  as  a  means  of  determining  air 
superiority,  since  every  Red  air-to-air  loss  corresponds  to  a  Blue  air-to-air  kill.  Those 
packages  which  are  considered  strong  in  the  air-to-air  role  can  be  expected  to  score  high 
in  this  category.  Table  3  summarizes  Red  air-to-air  losses  data  between  the  two  scenarios 
for  all  packages. 
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Table  3.  Red  Air-to-air  Losses 


Scenario  One 

Scenario  Two 
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In  scenario  one,  packages  one  and  three  were  statistically  in  a  group  by 


themselves  with  the  most  air-to-air  kills.  Packages  two,  four,  and  five  made  up  the  other 


group  with  statistically  fewer  air-to-air  kills. 

The  same  groupings  occurred  in  scenario  two,  although  the  means  were  slightly 
lower  for  each  package  that  experienced  a  Red  air-to-air  loss  in  scenario  one.  When  these 
data  are  compared  with  those  from  Table  2  (BAAL),  it  is  evident  that,  at  least  for  the  first 
seven  days  of  the  conflict,  the  loss/kill  ratio  for  the  Blue  forces  is  not  encouraging. 


Blue  SAM  Losses 


Air  defense,  or  SAM  (surface  to  air  missile),  losses  for  the  Blue  forces  served  as  a 


means  to  determine  the  effect  of  the  increased  nmnber  of  air  defenses  under  scenario  two 


and  the  effectiveness  of  the  SEAL)  missions.  The  data  are  summarized  in  Table  4. 


Table  4.  Blue  SAM  Losses 
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In  scenario  one,  packages  two  and  one  composed  the  group  with  the  most  losses. 


Packages  four,  five,  and  three  constituted  the  other  homogeneity  group  with  fewer  losses. 
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Scenario  two  resulted  in  the  same  groupings  as  scenario  one.  Based  on  the  mean 
and  standard  deviation  of  the  difference  between  packages  one  and  two  from  the  first 
thirty  replications,  it  was  determined  that  only  two  additional  runs  for  those  two  packages 
would  be  needed  to  separate  them  into  separate  groups.  After  an  additional  fifteen  runs 
however,  the  mean  difference  decreased  from  2.13  to  1.38.  This  change  resulted  in  an 
increase  in  the  number  of  runs  required  to  conclude  a  difference  exists  between  packages 
one  and  two  to  77. 

Packages  three,  four,  and  five  made  up  the  group  with  fewer  losses  on  average 
than  packages  one  and  two,  but  no  difference  could  be  found  between  the  three  packages. 
For  all  packages  in  both  scenarios,  there  were  considerably  more  Blue  losses  to  enemy  air 
defense  sites  than  those  from  enemy  aircraft.  Additionally,  doubling  the  number  of  air 
defense  sites  only  fractionally  increased  the  number  of  SAM  losses. 

FLOT  Movement 

FLOT  movement  is  the  distance  in  kilometers  that  the  front  line  on  the  groxmd 
moves  as  a  result  of  the  war  in  the  air.  This  is  a  vital  statistic  since  the  success  of  a 
campaign  is  usually  based  on  the  outcome  of  a  ground  war.  Data  describing  the  FLOT 
movement  are  shown  in  Table  5. 


Table  5.  FLOT  Movement 


Scenario  One 
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The  means  represent  cumulative  movement  for  the  duration  of  the  seven-day 
period  that  the  AEF  is  active.  The  negative  values  in  the  table  imply  that  the  Blue  forces 
are  being  pushed  back  on  the  ground.  We  would  expect  negative  values  for  the  first  few 
days  of  a  conflict  since  it  is  not  likely  that  an  enemy  would  be  pushed  back  immediately 
after  the  arrival  of  Blue  forces  in  theater. 

No  significant  statistical  differences  were  foimd  between  the  packages  in  either  of 
the  scenarios.  This  attributed  to  the  fact  that  there  was  little  variation  in  PLOT  among 
each  of  the  SnPm  combinations.  This  resulted  in  few  nonzero-valued  differences 


between  packages,  therefore  low  standard  deviations  (the  largest  standard  deviation 
among  the  differences  of  all  SriPm  combinations  was  0.61)  and  means  of  differences 
between  packages.  However,  scenario  two  resulted  in  slightly  less  ground  loss  than 
scenario  one  for  all  packages. 

Blue  Air-to-ground  Kills 

Blue  air-to-ground  kills  (BAGK)  measured  how  effectively  each  package 
impacted  the  enemy’s  ground  resources.  The  data  in  Table  6  summarize  BAGK  results. 


Table  6.  Blue  Air-to-ground  ICills 
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In  scenario  one,  packages  four  and  five  made  up  the  group  with  the  most  air-to- 
ground  kills.  All  other  packages  were  in  groups  by  themselves,  indicating  that  this  metric 


was  sensitive  to  the  changing  package  composition.  After  the  first  thirty  replications,  no 
statistical  difference  was  evident  between  packages  two,  four  and  five  in  scenario  two. 

No  difference  was  expected  to  be  found  between  packages  four  and  five  since  the  values 
for  each  replication  were  identical  for  these  two  packages.  However,  based  on  the  mean 
and  standard  deviation  of  the  differences  from  the  first  thirty  replications,  package  two 
was  shown  to  be  statistically  different  from  packages  four  and  five  with  only  nine 
additional  replications. 

Table  6  represents  the  results  of  the  additional  replications.  As  stated  above,  no 
difference  exists  between  packages  four  and  five  in  either  scenario.  In  descending  order 
for  both  scenarios,  packages  two,  three,  and  one  were  the  sole  members  of  the  other  three 
homogeneity  groups.  For  all  packages,  there  were  more  ground  kills  for  the  scenario  with 
more  air  defenses  (scenario  two).  This  result  is  contrary  to  one  possible  logical 
expectation  that  there  would  be  fewer  ground  kills  in  areas  with  more  air  defense  sites. 
Blue  Maintenance  Activities 


The  number  of  required  unscheduled  maintenance  activities  on  Blue  aircraft 
(BMX)  reflects  the  degree  of  maintainability  of  each  package,  which  affects  combat 
capability.  These  data  are  shown  in  Table  7. 


Table  7.  Blue  Maintenance  Activities 
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In  scenario  one,  packages  four  and  five  made  up  the  group  which  required  the 
higher  number  of  maintenance  activities.  Packages  one,  two,  and  three  made  up  the  other 
group  requiring  fewer  maintenance  actions. 

Scenario  two  resulted  in  the  unusual  situation  of  package  three  being  a  member  of 


both  groups,  allowing  for  no  conclusions  to  be  drawn  about  that  particular  package. 
Package  one  was  shown  to  require  the  fewest  number  of  unscheduled  maintenance 
actions  of  all  packages.  Packages  two,  four  and  five  experienced  the  highest  number  of 
maintenance  activities. 

Short-tons  Required 

This  measure  gives  an  estimate  of  the  amount  of  equipment  needed  to  support  a 
given  package.  The  logistics  measures  required  for  this  study  were  not  available  from 
THUNDER  since  the  simulation  precedes  the  AEF  concept  of  small,  lethal  forces 
deployed  for  a  short  time.  The  data  used  in  this  study  are  best  estimates  currently 
available  fi’om  the  Logistics  Plans  commvmity,  since  there  has  been  no  rapid  response 
AEF  deployment  on  which  to  base  any  logistics  measures.  Due  to  the  nature  of  the  data, 
only  point  estimates  are  available.  Statistical  procedures  as  previously  mentioned  do  not 
apply  to  these  data.  The  amount  of  equipment  required,  in  short-tons,  for  each  package 
appear  in  Table  8. 


Table  8.  Short-tons  Required 
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The  figures  are  identical  for  each  package  for  scenario  one  and  scenario  two 
because  no  significant  changes  in  the  required  equipment  would  be  made  based  on  the 
scenarios  as  defined  in  this  study.  A  factor  such  as  climate  could  cause  tailoring  of  the 
applicable  UTC  from  scenario  to  scenario,  if  a  deployment  to  location  with  an  extreme 
(hot,  cold,  wet,  dry,  etc.)  climate  is  imminent.  Drastic  changes  in  the  enemy  threat  could 
perhaps  prompt  a  UTC  review  before  a  deployment,  but  not  on  the  scale  as  was 
experienced  between  the  scenarios  used  in  this  study. 

Package  five  was  created  as  a  notional  package  for  this  study  as  an  attempt  to 
perform  a  sensitivity  analysis  on  package  four,  the  other  single  MDS  AEF.  The  large 
discrepancy  between  the  first  four  packages  and  package  five  is  due  to  the  fact  that  since 
package  five  is  not  currently  being  considered  for  an  AEF,  there  is  no  lead  unit 
distinction  for  any  of  the  squadrons  included  in  the  package.  A  lead  unit  is  one  which, 
during  an  AEF,  supplies  common  support  equipment  for  the  package.  The  lead  unit 
therefore  is  the  heaviest  unit  during  the  deployment. 

For  packages  one  through  four,  a  lead  unit  designation  is  made.  The  figures 
which  are  listed  for  these  packages  are  the  result  of  a  lead  unit  having  been  assigned. 
Because  package  five  has  never  deployed,  there  is  no  lead  unit  distinction.  Therefore,  the 
UTCs  which  served  as  the  source  for  these  data  considered  each  squadron  deploying 
independently  of  the  other  two,  so  all  three  are  listed  as  being  heavier  than  they  actually 
would  be. 

The  final  chapter  contains  a  summary  of  results  in  the  form  tabulated  rank 
orderings  for  each  of  the  MOEs  investigated.  Lessons  learned  will  also  be  offered  to  help 
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other  researchers  venturing  down  paths  similar  to  this  study.  The  chapter  will  close  with 
recommendations  for  further  research  into  this  area. 
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V.  Conclusions 


This  chapter  is  a  culmination  of  the  results  from  the  THUNDER  runs,  statistical 
procedures,  and  MOE  ranking  performed  up  to  this  point.  A  review  of  the  research 
questions  from  Chapter  1  will  be  conducted  and  conclusions  are  suggested  where 
statistical  support  is  available.  Recommendations  for  further  research  into  related  areas  is 
offered,  as  well  as  advice  for  those  who  seek  to  use  THUNDER  in  the  future. 

Before  introducing  results,  the  fact  that  these  data  were  produced  by  an 
unclassified,  notional  THUNDER  database  must  be  re-emphasized.  The  ramification  of 
using  the  Storm  database  is  that  the  performance  data  and  other  figures  contained  therein 
will  not  reflect  actual  classified  information.  However,  the  database  does  provide  the 
most  reliable  unclassified  data  available,  and  in  situations  where  actual  data  are  classified, 
Storm  provides  realistic  estimates. 

Research  Questions 

1.  What  are  the  rankings  for  the  five  AEF  packages  for  each  scenario  in  terms  of 
each  MOE? 

For  Blue  air-to-air  losses  in  both  scenarios,  packages  one  and  three  were  shown  to 
have  more  losses  than  two,  four,  and  five.  The  interesting  point  here  is  that  the  single 
(F-16)  MDS  packages  reported  fewer  losses  than  the  dual  MDS  packages  in  both  cases. 
The  F- ISC’s  greater  ordnance  capacity  suggests  it  should  have  a  higher  survivability  rate 
in  an  air-to-air  engagement  with  the  enemy.  The  validity  of  this  finding  should  be 
verified  by  exercising  the  five  AEF  packages  in  actual  (real-world)  THUNDER  scenarios. 
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When  looking  at  Red  air-to-air  losses,  packages  one  and  three  in  both  scenarios 
score  the  most  air-to-air  kills  of  the  Red  forces.  Packages  two,  four  and  five  had 
negligible  scores.  (Of  the  180  replications  of  the  simulation  involving  these  six  scenario- 
package  combinations,  only  three  Red  air-to-air  losses  were  recorded.)  The  fact  that, 
based  on  the  scenarios  as  defined  in  THUNDER  for  this  study,  those  scenario-package 
combinations  which  experience  the  most  losses  also  record  most  enemy  kills  lends 
evidence  to  suggest  that  these  combinations  engaged  in  more  air-to-air  encounters. 

There  are  several  possible  reasons  for  this  result.  Those  packages  which  were 
presumed  at  the  outset  of  this  research  to  be  less  combat  capable  could  be  perceived  by 
the  enemy  as  being  no  significant  threat,  or  at  least  not  significant  enough  to  risk 
resources  in  an  engagement. 

Another  possible  explanation  for  the  air-to-air  results  is  that  those  packages  which 
were  perceived  as  more  combat  capable  would  pose  a  more  viable  threat  to  the  enemy, 
and  therefore  the  enemy  would  be  willing  to  risk  resources  to  repel  the  threat. 
Examination  of  the  other  MOEs  will  give  a  better  idea  of  just  how  "combat  capable"  each 
package  is  and  therefore  the  perception  of  the  enemy  of  the  threat  each  pose.  If  those 
packages  which  have  high  losses  and  few  kills  in  air-to-air  combat  score  favorably  in 
other  MOEs,  a  campaign  analyst  may  accept  these  air-to-air  results. 

Packages  one  and  two  had  significantly  more  losses  to  SAMs  in  both  scenarios 
than  packages  three,  four,  and  five.  Knowing  this,  we  should  not  jump  to  the  conclusion 
that  one  package  is  particularly  more  vulnerable  than  another  to  enemy  air  defense 
systems.  This  could  be  the  result  of  similar  effects  described  above.  A  package  with 
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poor  air-to-ground  performance  would  be  less  likely  to  attempt  to  penetrate  enemy  air 
defenses  and  attack  critical  groimd  targets,  and  therefore  be  less  susceptible  to  SAM 
losses.  Alternately,  a  package  which  sustains  high  air  defense  losses  may  also  be  the 
most  aggressive.  Such  a  package  may  be  more  capable  in  an  air-to-ground  role  and 
consequently  be  more  susceptible  to  air  defense  systems.  As  before,  we  must  look  at  all 
measures  before  we  make  any  conclusions  about  overall  effectiveness. 

No  conclusions  can  be  made  about  which  air  package  allows  for  a  quicker  halt  of 
enemy  ground  forces.  For  all  packages  in  both  scenarios,  the  average  PLOT  movement 
over  all  replications  ranged  from  -17.00  km  to  -16.83  km.  Differences  between  packages 
were  too  small  for  any  conclusions  to  be  justified. 

The  Blue  air-to-ground  kills  measure  is  perhaps  the  most  critical.  It  is  this  metric 
that  most  directly  measures  the  impact  of  an  air  package  on  the  enemy's  ability  to  wage 
war.  It  is  this  quantity  which  allows  war  planners  to  justify  the  high  losses  as  described 
earlier.  The  relative  rankings  between  the  packages  were  identical.  Within  each  scenario, 
packages  four  and  five  recorded  the  most  air-to-ground  kills.  In  descending  order 
packages  two,  three,  and  one  rounded  out  this  measurement. 

Perhaps  the  most  surprising  result  comes  from  this  metric.  The  single  MDS 
packages  had  the  most  air-to-ground  kills  in  both  scenarios.  Alternately,  package  one,  the 
package  consisting  of  F-15Cs  in  the  air  to  role  and  F-15Es  in  the  air-to-ground  role, 
recorded  the  fewest  air-to-groimd  kills  in  each  scenario.  This  result  runs  counter  to  the 
expectation  that  package  one  is  the  most  effective  due  to  the  F-15E’s  ability  to  carry  more 
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ordnance  than  the  Block  40  F-16.  As  with  the  Blue  air-to-air  losses  measure,  this  result 
requires  verification  in  a  real-world  THUNDER  scenario. 

The  final  measure  obtained  from  THUNDER,  Blue  maintenance  activities,  gives 
an  idea  of  the  supportability  of  each  package.  The  fewer  unscheduled  maintenance 
activities  a  package  requires,  the  more  reliable  the  package  can  be  said  to  be.  This  is  the 
only  MOE  that  differed  between  the  two  scenarios.  In  scenario  one,  packages  four  and 
five  required  more  maintenance  activities  than  packages  one,  two,  and  three. 

The  results  for  scenario  two  were  not  as  clear-cut  as  those  for  scenario  one.  The 
only  conclusion  that  could  be  drawn  was  that  package  one  required  fewer  maintenance 
activities  than  packages  two,  four,  and  five.  These  results  also  run  against  common 
intuition.  A  logical  assumption  is  that  the  smaller,  single-engine  aircraft,  the  F-16,  would 
require  less  maintenance  than  the  F-15,  a  larger,  twin-engine  aircraft. 

The  last  measure  investigated  was  short-tons  of  equipment  required  to  support  the 
deployment.  Of  the  original  four  packages  submitted  for  study  by  the  Battlelab,  these 
figures  matched  what  was  expected.  The  fifth  package,  the  one  which  was  created  solely 
for  the  purposes  of  this  study,  will  be  analyzed  separately  due  to  the  nature  of  how  these 
logistics  data  were  obtained,  as  described  in  the  previous  chapter. 

The  lightest  package  to  deploy  was  the  single-MDS  package  (package  four). 
Packages  two  and  three  were  the  next  lightest  to  deploy,  with  only  1.6  short-tons 
difference  (an  insignificant  amount)  between  these  two  packages.  Package  one  proved  to 
be  the  heaviest  to  deploy.  These  findings  do  appear  to  support  the  assumption  that  the 
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single  MDS  package  (package  four)  would  be  the  lightest  to  deploy,  due  the  ability  to 
cross-utilize  much  of  the  support  equipment  that  would  accompany  the  lead  unit. 

We  would  also  expect  package  two  and  three  to  be  the  next  lightest,  and 
somewhat  similar  to  each  other.  This  is  because  these  packages  are  very  similar  to  each 
other — ^they  both  consist  of  two  F-16  squadrons  and  one  F-15  squadron.  If  the  F-16 
squadrons  share  resources  and  the  F-15  squadron  deploys  independently  (and  hence  to  a 
large  degree  is  self-sufficient)  for  both  packages,  a  large  difference  between  the  two 
packages  should  prompt  questions. 

Package  one  was  expected  to  be  the  heaviest  to  deploy.  This  is  because  it  consists 
of  two  F-15  squadrons  and  one  F-16  squadron.  There  are  two  factors  which  contribute  to 
this  outcome.  First,  although  it  was  discussed  earlier  that  this  package  was  among  those 
which  required  the  fewest  imscheduled  maintenance  activities  according  these 
THUNDER  scenarios,  it  is  likely  that  more  support  equipment  would  be  sent  to  a 
deployment  involving  the  larger,  twin-engine  F-15. 

The  second  reason  package  one  would  be  considered  the  heaviest  to  deploy  is  that 
the  deployed  F-16  squadron  would  have  to  be  self-sufficient.  Although  cross-utilization 
does  exist  between  the  F-15  and  the  F-16  for  some  parts  and  support  equipment,  it  is  not 
likely  that  an  F-16  squadron  would  plan  to  deploy  in  this  manner  with  an  F-15  squadron. 
This  would  result  in  a  heavier  deployment  for  the  Block  50  F-16  squadron. 

It  was  mentioned  earlier  that  package  five  would  be  analyzed  separately  due  to  the 
method  used  to  obtain  these  data.  This  method  resulted  in  a  disproportionately  high 
figure  for  equipment  requirements.  If  this  package  is  ever  considered  as  a  viable  AEF 
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option,  the  logistics  plans  community  should  begin  studying  its  logistics  requirements 
and  create  documents  similar  to  those  used  to  obtain  data  for  the  other  packages  in  this 
study.  It  is  expected  that  this  package  would  require  somewhat  more  support  than 
package  four,  but  less  than  all  the  other  packages. 

2.  Would  increasing  the  number  of  aircraft  in  the  lightest  package  increase  that 
package ’s  combat  output? 

In  none  of  the  ten  the  scenario-package  combinations  did  THUNDER  produce  a 
significant  difference  between  packages  four  and  five.  In  fact,  eight  of  the  ten  means 
were  foimd  to  be  identical.  This  finding  supports  the  notion  that  when  THUNDER  is 
used  to  compare  alternatives,  such  as  in  this  case,  results  are  more  sensitive  to  changes  in 
the  percent  of  total  force  composition  that  a  particular  weapon  system  attains  as  opposed 
to  keeping  proportions  constant  and  changing  in  the  number  of  aircraft  in  the  package. 

3.  Is  the  relative  ranking  of  combat  effectiveness  of  each  air  package  constant 
throughout  different  threat  scenarios? 

The  only  MOE  that  lead  to  different  conclusions  about  the  ranking  of  packages 
between  scenarios  was  Blue  maintenance  activities.  In  scenario  one,  packages  four  and 
five  were  found  to  require  more  unscheduled  maintenance  than  packages  one,  two,  and 
three.  In  scenario  two,  packages  two,  four  and  five  were  found  to  require  more 
maintenance  than  package  one.  No  conclusions  could  be  drawn  concerning  the  ranking 
of  package  three.  For  all  other  MOEs,  conclusions  regarding  the  rankings  of  the 
packages  in  scenario  one  were  identical  to  those  in  scenario  two. 
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It  is  presumed  that  varying  the  enemy  threat  to  a  larger  degree  than  that  which 
was  performed  in  this  study  should  result  in  changes  to  the  ranking  of  the  packages  for 
various  MOEs.  The  fact  that  the  numbers  of  both  enemy  air  defenses  and  aircraft  were 
drastically  reduced  to  produce  scenarios  which  resulted  in  meaningful  data  may 
contribute  to  the  similarities  found  between  scenarios.  Scaling  down  the  enemy  threat  to 
the  degree  that  was  required  in  this  study  reduced  the  margin  of  allowable  changes 
between  scenarios.  A  larger  enemy  threat  would  allow  for  larger  changes  in  that  threat, 
and  ultimately  perhaps  different  rankings  between  scenarios.  However,  a  threat  that  is 
large  enough  to  modify  in  such  a  way  that  would  result  in  changing  the  ranking  of  the 
packages  between  scenarios  would  likely  be  too  large  for  an  AEF  as  defined  in  the  study. 
Recommendations  for  Further  Research 

The  difficulty  in  determining  the  required  support  for  each  package  was  a 
tremendous  obstacle  to  overcome.  There  currently  appears  to  be  no  central  location  for 
this  information.  The  AEF  concept  has  not  yet  evolved  to  the  point  where  there  is  a 
consensus  for  required  support  for  each  package.  The  Eagle  and  Falcon  packages 
(packages  one  and  three  respectively  in  this  study)  have  received  much  attention  and  data 
for  these  packages  are  available. 

However,  the  other  packages  in  this  study  appear  to  be  operationally  feasible. 
Extrapolating  from  known  data  from  the  Eagle  and  Falcon  packages  in  an  attempt  to  “fit” 
data  to  the  other  packages  was  the  only  option  available  to  this  researcher.  However, 
assumptions  relating  to  the  lead  unit  may  not  transfer  from  package  to  package.  The 
logistics  regarding  packages  two,  four,  and  five  are  worthy  of  investigation. 
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It  was  mentioned  earlier  that  the  reader  should  keep  in  mind  that  the  results  from 
the  THUNDER  runs  in  this  study  were  the  products  of  an  unclassified,  notional  database. 
Changing  the  database  to  one  that  depicts  an  actual  threat  could  change  the  results  of  this 
study.  However,  the  methodology  and  data  analysis  contained  herein  are  applicable  to 
any  database  that  may  be  used.  The  researcher  interested  in  continuing  this  work  should 
consider  running  these  packages  through  an  actual  (real  world)  scenario.  Considering  a 
deployment  to  another  location  should  also  be  investigated.  This  study  examined  a 
Middle  East  scenario.  Deployments  to  other  locations  should  also  be  considered. 

Another  area  of  research  utilizing  THUNDER  involves  advances  in  weaponry. 

A  briefing  presented  by  the  RAND  Corporation  discussed  earlier  in  the  literature  review 
introduces  a  “small-smart”  mimition.  With  this  new  air-to-ground  ordnance,  a  250- 
pound  bomb  containing  42  pounds  of  a  revolutionary  high-explosive  is  expected  to 
generate  the  same  explosive  power  as  the  traditional  Mk-84  2000-pound  bomb  containing 
945  pounds  of  high-explosive.  This  advancement  alone  is  expected  to  reduce  the  current 
need  of  thirty-two  C-141  equivalents  for  munitions  to  four  (Tripp,  1997). 

A  database  including  these  munitions  could  be  built  with  the  desired  result  being 
to  discover  how  lighter,  equally  destructive  munitions  affect  the  overall  outcome.  If  the 
strike  aircraft  have  enough  suspension  points  to  deploy  more  of  these  munitions,  the  air- 
to-ground  results  would  be  expected  to  inerease. 

Another  area  of  research  involving  these  munitions  and  THUNDER  would  be 
studying  explosive  power  versus  amoimt  of  high-explosive  and  estimate  the  actual 
circular  error  probability  (CEP)  of  a  Mk-84  equivalent  bomb  containing  this  new 
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explosive.  This  different  (increased)  CEP  could  be  programmed  into  THUNDER  while 
keeping  all  other  parameters  constant.  If  this  avenue  of  study  is  explored,  the  number  of 
bombs  a  strike  aircraft  could  employ  would  be  the  same  as  before,  but  each  bomb  would 
be  more  powerful.  The  result  in  this  case  may  be  fewer  required  strikes  to  destroy 
hardened  targets. 

An  interesting  finding  was  that  THUNDER  output  appears  to  be  more  sensitive  to 
the  types  of  aircraft  as  opposed  to  number  of  aircraft  that  make  up  a  given  package. 
Packages  four  and  five  have  identical  mixes  of  aircraft,  and  vary  from  each  other  only  in 
the  number  of  aircraft.  These  packages  have  similar  output  for  each  of  the  measures  even 
though  package  five  has  50  percent  more  aircraft  than  package  four.  Further  research 
should  be  conducted  to  determine  how  many  more  aircraft  need  to  be  added  to  package 
four  to  obtain  statistically  different  output.  Further  research  should  also  be  conducted  on 
different  packages  (not  necessarily  those  investigated  herein)  to  see  if  this  phenomenon  is 
present  in  other  force  mixes. 

Advice  to  Future  THUNDER  Users 

A  training  class  is  offered  by  831  (System  Simulation  Solutions,  Inc;  the 
contractor  who  currently  maintains  THUNDER)  in  Alexandria,  Virginia  to  help  analysts 
utilize  thunder’s  full  capabilities.  This  class  is  an  invaluable  asset  to  those  who  will 
be  using  THUNDER  regularly.  There  has  not  been  a  need  for  students  in  the  School  of 
Logistics  and  Acquisitions  Management  to  attend  this  class  in  the  past.  However,  those 
students  in  the  School  of  Engineering  who  will  be  using  THUNDER  do  attend  the 
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training.  This  class  would  certainly  aid  any  student  who  wishes  to  use  THUNDER, 
whether  for  a  thesis  or  for  their  next  assignment. 

If  a  student  uses  THUNDER  for  a  study  such  as  this  and  does  not  attend  the 
training  class,  their  efforts  will  be  greatly  hampered.  But  before  being  able  to  even  use 
THUNDER,  the  student  must  have  knowledge  of  basic  UNIX  commands.  Depending  on 
the  experience  of  the  student,  several  months  of  trial  and  error  may  be  required  in  order  to 
obtain  emy  data  at  all,  let  alone  the  volume  required  for  a  study  such  as  this.  Even  after 
completing  the  runs  and  obtaining  raw  data,  the  student  still  must  know  how  to  process 
that  data  to  produce  usable  information.  This  requires  an  understanding  of  UNIX. 

If  the  student  decides  to  use  THUNDER,  it  is  best  to  develop  a  network  of 
contacts  so  as  not  to  overly  task  one  or  two  people.  As  stated  before,  there  are  several 
people  at  AFIT  who  have  used  THUNDER.  But  there  are  several  external  agencies  that 
also  use  THUNDER.  The  two  which  were  contacted  the  most  in  this  effort  were  the 
Aeronautical  Systems  Center  (ASC)  and  the  Air  Force  Studies  and  Analysis  Agency 
(AFSAA).  831  has  employees  contracted  out  to  ASC  to  do  analysis  work.  The  central 
location  for  military  THUNDER  use  is  AFSAA.  Specific  contacts  used  in  this  effort 
appear  in  the  Acknowledgements  section.  Throughout  this  study,  every  effort  was  made 
to  deal  first  with  active  duty  military  personnel  either  at  AFIT  or  AFSAA  before  talking 
to  a  contractor.  Other  points  of  contact  can  be  established  from  the  Acknowledgements 
page  of  other  theses  which  utilized  THUNDER. 
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Appendix  A:  Results  for  Each  Scenario-Package  Combination 

Output  data  of  each  SwP/w  for  each  MOE  by  replication  given  by  THUNDER. 

Rep  -  Replication  number 

BAAL  -  Blue  air-to-air  losses 

RAAL  -  Red  air-to-air  losses 

BSAML  -  Blue  surface  to  air  (SAM)  losses 

PLOT  -  PLOT  movement  in  kilometers  (negative  values  imply  PLOT  is  moving  back,  i.e. 

Blue  forces  are  losing  ground) 

BACK  -  Blue  air-to-groimd  kills 

BMX  -  Number  of  Blue  unscheduled  aircraft  maintenance  activities 


Table  9.  Scenario  One,  Package  One  Results 


Table  10.  Scenario  One,  Package  Two  Results 


Table  11.  Scenario  One,  Package  Three  Results 


Table  12.  Scenario  One,  Package  Four  Results 


Table  13.  Scenario  One,  Package  Five  Results 


Table  14.  Scenario  Two,  Package  One  Results 


Table  15.  Scenario  Two,  Package  Two  Results 


Table  16.  Scenario  Two,  Package  Three  Results 
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Table  17.  Scenario  Two,  Package  Four  Results 


Table  18.  Scenario  Two,  Package  Five  Results 


Appendix  B:  Summary  of  Additional  Replications 

Table  19.  Blue  Air-to- Air  Losses, 
Packages  Two  and  Five  in  Scenario  One 


SD(1-30):  1.196  SD  (1-60):  1.343 

LLCI:  -0.1  LLCI:  -0.28 

ULCI:  1.029  ULCI:  0.613 


Contains  0?  YES  Contains  0?  YES 

Required  n:  44  Required  n:  431 

Additional  reps?  YES  Additional  reps?  NO 


Table  20.  Blue  SAM  Losses, 
Packages  One  and  Two  in  Scenario  Two 


SD(1-30):  4.652 
LLCI:  2.133 
ULCI;  2.133 
Contains  0?  NO 
Required  n;  30 
Additional  Reps?  NO 


SD(1-45):  4.687 


LLCI:  -0.42 
ULCI:  3.177 
Contains  0?  YES 
Required  n:  77 
Additional  reps?  NO 


Table  2 1 .  Blue  Air-to-ground  Kills, 
Packages  Two  and  Four  in  Scenario  Two 


SD(1-30):  87.06 
LLCI:  -77.3 
ULCI:  4.597 
Contains  0?  YES 
Required  n;  39 
Additional  Reps?  YES 


SD(1-45):  96.47 


LLCi:  -76.5 
ULCi:  -2.39 
Contains  0?  NO 
Required  n:  45 
Additional  Reps?  NO 


Table  22.  Blue  Air-to-ground  Kills, 
Packages  Two  and  Five  in  Scenario  Two 


iJimariBcmiraci 


SD(1-30):  87.06 
LLCI:  -77.3 
ULCI:  4.597 
Contains  0?  YES 
Required  n:  39 
Additional  Reps?  YES 


SD(1-30):  96.47 
LLCI;  -76.5 
ULCI:  -2.39 


Contains  0?  NO 
Required  n:  45 
Additional  Reps?  NO 
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